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The  real-time  automated  registration  of  multi-sensor  imagery  begins  with  the  generation  of  control 
information.  A  specific  application  may  require  the  registration  of  newly  acquired  data  to  an  existing 
spatial  database  (absolute  registration),  or  to  other  images  of  a  series  (relative  registration).  This  study 
examines  the  the  feasibility  and  upper-level  design  of  a  system  capable  of  providing  the  control 
information  required  for  a  range  ol  image  registration  tasks  ami  image  types.  In  general,  the  control 
generator  we  suggest  will  be  guided  by  a  spatial  database  maintaining  information  about  the  feature 
content  of  the  area  of  interest.  A  rule-based  query  generator  will  extract  candidate  ground  control 
optimized  for  the  particular  image  type  and  geometry  at  hand. 


Summary 


This  document  is  the  final  report  foi  Phase  I  of  an  investigation  into  the  feasibility  and  design  of  a  smart 
control  generator  for  the  purpose  of  automated  image-to-map  and  image -to- image  registration.  The  basic 
issues  examined  in  this  report  are: 


*  Characteristics  of  a  spatial  database  specific  to  the  control  generation  task. 

*  The  utility  of  various  existing  spatial  data  structures  with  respect  to  control  generation. 

*  The  top-level  design  of  a  prototype  system  to  be  used  for  further  analysis  and  experimentation  in 
automated  control  generation. 
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Introduction 


Phase  I  was  initially  proposed  to  address  the  high-level  concepts  of  an  autonomous  system  for  the  registration 
of  images  and  spatial  map  databases.  The  implementation  of  an  autonomous  image  registration  system  on  a 
digital  computer  has  been  one  of  the  initial  and  continuing  goals  of  research  in  the  exploitation  of  remotely 
sensed  data.  Because  the  registration  of  multiple  images  is  a  prerequisite  task  of  virtually  any  use  of  those 
images  for  map  creation  and  updating  or  object  detection  and  tracking,  techniques  that  can  aid  a  human 
operator,  or  actually  accomplish  the  registration  automatically,  are  of  great  importance. 

The  work  done  during  Phase  I  was  the  result  of  a  redirection  of  our  efforts  to  concentrate  on  determining  the 
feasibility  of  using  a  digital  spatial  database  to  support  the  generation  of  ground  control  in  support  of 
automated  image  registration  procedures.  Additionally,  we  outline  the  specific  characteristics  desirable  in 
such  a  database.  It  was  determined  that  questions  of  feature  and  image  matching,  and  of  the  use  of  image 
data  for  registration  control  were  not  within  the  scope  of  Phase  I. 

The  Smart  Mapping,  Charting  and  Geodesy  (MC&G)  Control  Generator  will  extract  ground  control 
information  from  this  spatial  database  system.  For  the  current  study  it  was  assumed  that  the  application  was 
not  in  a  real-time  setting  -  typically  operating  in  an  interactive,  operator-assisted  environment.  This  is  in 
contrast  to  a  real-time  application  such  as  battlefield  surveillance  and  management. 

Our  study  of  the  use  of  feature  information  extracted  from  a  spatial  database  leads  us  to  conclude  that  such 
information,  as  it  currently  exists,  is  not  adequate  to  the  task  of  automated  image  registration.  These  existing 
databases  of  feature  information  are  neither  accurate  nor  complete  enough  to  be  relied  on  as  a  sole  source  of 
ground  control.  Even  a  complete  and  accurate  spatial  database  would  not  be  an  appropriate  source  of  control 
information  for  some  registration  tasks.  Additional  sources  of  control  information  will  be  necessary, 
particularly  in  a  real-time  setting.  Possible  sources  for  control  leading  to  a  refinement  of  the  registration 
process  include  various  dead-reckoning  tools  and  image-to-imnge  matching. 

We  find  that,  in  general,  the  function  of  choosing  candidate  ground  control  information  and  methods  is  an 
aspect  of  autonomous  registration  that  can  potentially  be  automated  with  current  technology.  The 
characteristics  of  imagery  and  matching  techniques  that  lead  to  control  selection  are  knowledge  of  a  sort  that 
can  be  codified  and  manipulated  with  existing  software  techniques.  Modeling  of  the  location  errors  inherent 
in  the  various  feature  and  matching  techniques  and  in  the  coordinate  transformations  involved  in  registration 
can  be  performed  to  allow  the  determination  of  control  appropriate  for  differing  registration  tasks. 


The  creation  ami  maintenance  of  digital  map  databases  is  a  subject  of  growing  importance  in  both  military 
and  commercial  endeavors.  The  use  of  various  types  of  remotely  sensed  imagery  in  updating  and  exploiting 
such  databases  must  begin  with  the  currently  labor-intensive  and  lime  consuming  task  of  image  registration. 
Thus,  techniques  to  automate  the  control  generation  and  registration  processes  will  greatly  enhance  the  utility 
of  digital  map  databases  for  numerous  applications.  Among  the  existing  and  possible  uses  for  large-scale  map 
databases  from  remotely  sensed  data  are: 

Forestry  and  land  management 

Geological  analysis  and  prospecting 

Military  applications  (battlefield  surveillance,  target  identification  and  tracking,  etc.) 

Municipal  land-use  planning 

Scientific  research  (botany,  oceanography,  geology,  planetology,  etc.) 

Virtually  any  use  that  has  been  found  in  the  past  for  maps  and  map-cori elated  data  will  benefit  from  the 
eventual  wide-spread  existence  of  techniques  for  the  automatic  creation,  updating  and  exploitation  of  digital 
map  databases.  Thus,  the  Smart  Control  Generator  will  be  an  essential  part  of  a  technology  certain  to  be  of 
great  importance  in  the  near  and  far  future. 

The  Phase  I  effort  is  concerned  primarily  with  a  feasibility  analysis  of  the  registration  of  imagery  not  to  other 
images,  but  to  a  digitally  stored  representation  of  the  region  or  surface  imaged.  This  representation  is  in  the 
form  of  spatial  databases  or  geographical  information  systems  (GISs).  Image  registration  requires  that 
ground  control  points  and  features  be  extracted  from  the  database  containing  the  spatial  representation  and 
passed  to  additional  software  that  will  attempt  to  precisely  locate  those  points  and  features  in  the  actual  image 
of  interest.  Thus,  although  our  primary  concern  is  to  analyze  the  various  techniques  for  digitally 
representing  spatial  data,  actual  implementation  of  such  a  system  must  attend  to  the  specific  feature 
identification  and  matching  techniques  to  be  used  with  the  imagery  in  question. 

Typically,  the  registration  task  begins  with  a  human  interpreter  examining  an  image  or  map  for  features  and 
patterns  that  s/he  expects  to  be  visible  in  the  image  to  be  registered.  These  expectations  are  based  on 
accumulated  experience  with  the  imagery  in  question  as  well  as  the  theoretical  characteristics  of  the  sensor 
technology.  Additionally,  knowledge  of  the  sensor  geometry  and  surface  topography  is  used  to  constrain  the 
area  searched  for  ground  control  information.  The  purpose  of  a  Smart  Mapping,  Charting  and  Geodesy 
(MC&G)  Control  Generator  is  to  automate  these  reasoning  processes.  The  human  operator  chooses  ground 
control  knowing  that  the  mechanism  for  feature  discovery  and  correlation  will  be  human  vision.  Because  in 
an  autonomous  image  registration  system  these  matching  tasks  will  be  implemented  by  the  artificial 
(computer)  analogues  of  human  vision,  the  control  generation  system  most  account  for  (he  nature  of  those 
analogues. 


The  development  of  an  automated  control  generation  process  basically  requires  the  quantitative  modeling  of 
the  registration  procedure.  The  characteristics  of  the  various  aspects  of  the  procedure  must  be  known  in 
order  to  predict  the  control  features  and  methods  most  suitable  to  a  given  registration  task.  These  aspects 
include  the  initial  sensor  position  determining  techniques,  the  storage  of  feature  data,  the  matching 
procedures  and  the  nature  of  the  task.  Characteristics  about  each  of  these  aspects  include  the  input  data  or 
situations  required,  their  performance  with  respect  to  the  specific  task,  their  accuracies  (and  associated 
possibilities  of  introducing  error),  etc..  The  process  of  developing  a  Smart  MC&G  Control  Generator  will 
require  a  system  to  accurately  describe  the  automated  registration  process  as  it  can  be  currently  performed. 
Additionally,  the  system  should  retain  the  flexibility  to  permit  consideration  of  advances  in  any  and  all  of  the 
various  aspects  of  that  process. 

The  database  structure  we  find  to  be  most  appropriate  to  control  generation  has  a  highly-structured  arc/node 
format.  This  provides  advantages  of  storage  space  conservation  and  multiple  levels  of  data  organization. 
Some  capabilities  this  database  should  exhibit  include: 

*  geographical  ’clustering’  of  features  in  physical  memory; 

*  similar  clustering  of  features  according  to  their  size; 

*  memory-resident  operations  to  allow  feature  examination  and  modification  with  minimal  disk 
access; 

*  object-oriented  representation  of  features. 

There  exist  spatial  database  systems  that  possess  these  attributes  and  others  that  make  them  suitable  for 
further  work  with  automated  control  generation.  One  of  them,  the  Kork  Geographical  Information  System 
from  Kork  Systems,  Inc.,  is  available  to  VEXCEL  and  is  proposed  as  the  database  system  to  be  used  for 
further  work  in  this  area.  Details  about  our  examination  of  existing  spatial  database  techniques  will  be 
discussed  later. 

The  construction  of  a  rule-based  system  for  extracting  relevant  information  from  the  database  will  require  the 
greatest  portion  of  the  effort  of  implementing  Smart  MC&G  Control  Generation  functions  using  spatial 
databases.  It  is  possible  only  because  the  task  of  feature  choice  for  ground  control  is  well-constrained  by  the 
physical  parameters  of  the  sensors  and  image  matching  methods  used.  The  control  generator  will  also 
require  an  integrated  elevation  (terrain)  modeling  capability  to  aid  in  restricting  the  geographical  region  to  be 
searched  for  control  information  and  in  the  construction  of  simulated  images.  These  functions  will  utilize 
existing  three-dimensional  data  maintained  in  the  spatial  database.  The  creation  and  addition  of  new  elevation 
data  will  not  be  an  aspect  of  the  initial  development  of  a  control  generator. 


Section  2 


Basic  Functionality 


The  functions  of  the  Smart  M C&G  Control  Generator  fall  into  six  basic  areas: 


*  the  geographical  (feature)  information  itself,  including  attributes  of  those  features: 

*  the  image  data  itself  (i.e.  a  library  of  image  segments): 

*  the  integration  of  initial  sensor  position  and  orientation  information  (i.e.  data  from  dead-reckoning 

tools  such  as  inertial  navigation  or  global  positioning  systems); 

*  the  intelligent  querying  of  the  information; 

*  the  elevation  (topographic)  information  processing; 

*  the  evaluation  and  reporting  of  the  discovered  potential  ground  control  points  (including  error 
estimation). 

The  first  three  of  these  items  constitute  the  information  to  be  managed,  while  the  last  three  are  the  specific 
management  functions  to  be  performed  for  this  application.  The  geographical  information  will  initially  be  a 
digital  representation  of  existing  maps;  specifically  of  the  features  present  on  the  maps  that  are  likely  to 
appear  in  some  image.  This  includes  natural  features  such  as  streams,  lakes  and  forests  as  well  as  cultural 
features  like  roads  and  buildings.  The  elevation  modeling  functions  are  considered  separately  because  the 
need  for  their  utilization  in  ground  control  generation  is  less  common  (and  more  expensive)  than  planimetric 
data.  Image  segments  can  be  maintained  in  a  companion  database,  or  as  objects  in  the  GIS  itself.  These 
segments  may  be  used  as  tools  to  predict  and  recall  feature  appearance  during  image  matching  procedures. 
Data  from  various  dead-reckoning  navigation  and  orientation  systems  can  serve  for  coarse  image  correlation. 
This  may  be  satisfactory  in  itself  for  some  applications,  or  may  be  useful  as  an  initial  registration  (subject  to 
refinement)  for  other  tasks. 

The  actual  application  of  this  data  to  automated  image  registration  involves:  first,  the  efficient  search  for 
ground  control  for  the  image  of  interest,  and  second,  the  estimation  of  the  probable  limits  of  error  between 
the  predicted  and  actual  location  of  the  ground  control  features  in  the  image.  A  high-level  diagram  of  the 
functions  performed  during  automated  image  registration  follows. 


Functional  Diagram 


Nole:  Dashed  lines  indicate  functions  not  always  necessarily  performed. 
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Section  3 


Operational  Scenario 


1 


What  follows  is  an  outline  of  some  of  the  operations  typically  performed  by  the  Smart  MC&G  Control 
Generator  when  given  the  task  of  reporting  candidate  ground  control  for  a  new  image: 

i 

1)  Image  characteristics  input  to  query  generator  1 

-  sensor  type; 

-  preliminary  sensor  geometry; 

-  environment  (weather). 

j 

2)  Delineation  of  the  search  area  by  application  of  dead-reckoning  information  > 

a)  Inertial  navigation,  satellite  epltemerides,  or  other  a  priori  knowledge  of  the  sensor  position  j 

and  orientation  is  used  to  eliminate  geographical  regions  not  visible  to  the  sensor  from  , 

I 

consideration  for  ground  control  generation.  j 

b)  This  information  is  also  used  in  the  derivation  of  error  limits  between  predicted  and  actual 
feature  positions  in  the  image. 


3)  Initial  query  generation 

a)  sensor  resolution  +  sensor  range  used  to  determine  feature  size  range  to  be  considered; 

b)  sensor  type  +  environment  used  to  determine  feature  types  and  attributes  to  be  searched  for; 

c)  look  angle  +  altitude  +  average  relief  4-  image  registration  technique  used  to  determine 

need  to  consider  DEM. 

4)  If  DEM  required 

a)  create  DEM  from  GIS  elevation  data  (if  not  already  computed  and  maintained); 

b)  create  perspective  view  (with  accuracy  as  specified  by  query  generator  in  3c  above)  to 

determine  regions  visible  to  sensor. 

5)  Extract  all  features  of  proper  type/attribute  within  visible  regions. 

6)  If  image  type  and  optimal  matching  process  requires  simulated  image 

a)  create  simulated  images  of  regions  useful  for  image-to-simulated-image  matching  and 
registration. 


7)  Evaluate  initial  set  of  candidate  control  features 

a)  secondary  effects 

-  orientation  with  respect  to  sensor/flightpath? 

-  small  features  hidden?  (refine  DEM) 

-  clutter; 

b)  too  few  features  (or  poorly  distributed)? 

c)  relax  parameters  and  re-query  if  necessary. 

8)  Calculate  error  limits  of  control  points 

-  from  map  accuracy  +  sensor  resolution  +  sensor  geometry  accuracy. 

9)  Pass  features,  locations  and  error  limits  to  image  matching  software. 

The  process  described  above  illustrates  the  use  of  the  major  functions  of  the  control  generator.  Other  queries 
concerning  ground  control  (e.g.  additional  information  about  a  specific  feature,  additional  features  for  refined 
registration,  etc.)  will  utilize  these  functions  under  the  control  of  the  rule-based  query  generation  and  system 
control  software. 


Section  4 


The  Smart  Mapping.  Charting  and  Geodesy  Control  Generator 

The  Smart  MC&G  Control  Generator  comprises  these  three  basic  modules: 

*  the  geographic  information  system  (GIS): 

-  feature  data  (map  database). 

-  image  data  (library  of  image  segments). 

*  integrated  digital  elevation  modeling  (DEM)  module; 

*  query  generation  and  system  control; 

-  intelligent  querying  of  GIS. 

•  choice  and  coordination  of  image  processing/matching  techniques. 

-  integration  of  dead-reckoning  techniques, 

-  reporting  of  candidate  control  and  error  estimates. 

By  separating  the  actual  data  storage  and  retrieval  functions  from  the  ground  control  query  and  analysis,  the 
system  is  more  easily  upgraded  as  new  feature  and  sensor  types  and  attributes  become  relevant.  Further,  the 
implementation  of  the  query  generator  as  a  distinct,  rule-based  system  will  allow  the  GIS  to  be  used  for  other 
applications,  requiring  only  that  the  appropriate  rule-set  be  created. 

The  control  generation  system  will  be  implemented  using  object-oriented  programming  techniques.  The  GIS 
will  support  feature  storage  and  access  as  objects.  Object-oriented  programming,  for  example  Smalltalk,  has 
its  roots  in  simulation,  and  consequently  has  as  a  fundamental  structure  the  notion  of  "object".  These 
objects  can  both  constitute  a  representation  of  something,  as  well  as  actually  do  something.  The  former 
property  corresponds  to  the  notion  of  an  "instance  variable",  while  the  latter  corresponds  to  the  notion  of  an 
"instance  method”.  Each  object  has  its  own  unique  set  of  instance  variables  and  instance  methods. 

Object-oriented  programming  also  has  the  capacity  for  hierarchical  classification.  Objects  belong  to  "classes”, 
which  in  turn  can  be  aggregated  to  form  "superclasses",  or  subdivided  into  "subclasses".  These  classes, 
subclasses,  and  superclasses  also  have  instance  variables  and  instance  methods.  One  of  the  conveniences  of 
these  hierarchical  constructs  is  the  property  of  "inheritance".  Objects  automatically  inherit  the  instance 
variables  and  instance  methods  of  the  class  to  which  they  belong.  Of  course,  these  may  be  modified  in 
individual  cases,  thereby  retaining  flexibility. 


For  the  purposes  of  the  map  information  database,  individual  features  could  be  represented  as  objects. 
Classes  of  objects  would  correspond  to  real-world  aggregations  of  objects  that  are  appropriate  to  imaging. 
Instance  variables  of  an  object  would  correspond  to  those  qualities  or  characteristics  that  are  appropriate  to 
imaging  with  a  particular  sensor  type.  Instance  methods  would  correspond  to  particular  methods  of  choosing 
control  information  or  particular  methods  of  forming  synthetic  images  for  a  particular  sensor  type. 

Therefore,  for  this  particular  application,  object-oriented  programming  allows  a  convenient,  flexible,  and 
hierarchical  definition  of  properties  for  map  features  which  arc  complementary  to  the  corresponding  definition 
of  imaging  processes  that  are  relevant  to  those  properties. 

\ 

Another  consideration  is  that  of  extensibility.  Any  instance  method  will  work  on  any  object  if  the  former  is 
included  in  the  latter’s  collection  of  instance  methods,  or  '’protocol”.  Therefore,  one  can  define  a  new  class 
and  allow  many  existing  methods  to  work  for  it  by  suitably  defining  its  protocol.  In  this  way.  new  map 
features  can  be  added  to  the  database  and  potentially  be  amenable  to  some  of  the  previously-defined  imaging 
and  control  information  processes. 

The  remainder  of  this  section  will  be  devoted  to  a  more  detailed  description  of  the  three  basic  modules  of  the 
control  generation  system. 

4.1  Geographic  Information  System 

Clearly,  the  heart  of  the  control  generator  is  the  database  itself.  The  basic  approach  to  registration  that  we 
consider  could  be  called  ”map-guided”.  The  existing  map  database  is  searched  for  possible  ground  control 
according  to  the  current  needs  of  the  registration  system  at  the  time.  Thus,  the  format  of  the  database  must 
be  optimized  for  the  sorts  of  queries  most  likely  to  be  made.  These  queries  can  be  divided  into  two  classes: 

*  search  by  geometry; 

*  search  by  attribute(s). 

Naturally,  queries  of  the  database  will  often,  if  not  always,  be  some  combination  of  these  two  types. 

It  should  be  clearly  understood  that  the  contents  of  the  database  may  consist  partially  of  terrain  features  that 
were  automatically  extracted  from  imagery.  This  is  in  addition  to,  or  as  opposed  to,  map-related  data. 


(a)  Geometric  Search 


A  common  need  will  be  a  search  for  all  features  (perhaps  with  specified  attributes)  within  a  certain  area  or 
within  a  certain  distance  of  a  given  point.  This  is  a  tvpical  function  of  most  Geographical  Information 
Systems  (GISs).  and  a  number  of  techniques  have  been  developed  to  facilitate  this  type  of  data  manipulation. 
To  speed  the  execution  time  of  such  queries,  the  digital  representation  of  features  near  each  other  in  the 
world  should  be  “near"  each  other  in  the  physical  structure  of  the  database.  The  purpose  of  this 
geographical  clustering  is  to  reduce  the  number  of  disk  accesses  needed  to  examine  a  given  area. 

Geographical  clustering  can  be  done  by  storing  as  a  feature  attribute  the  range  (minimum  and  maximum)  of 
coordinates  spanned  by  each  feature.  A  number  of  algorithms  exist  to  sort  these  Minimum  Bounding 
Rectangles  (MBRs)  so  that  all  features  within  a  given  region  can  be  quickly  accessed.  Among  these  is  an 
extension  of  B-trees  to  more  than  one  dimension,  called  an  R-tree  (for  rectangular). 

The  use  of  differing  sensor  technologies  (with  differing  resolutions)  creates  a  requirement  not  so  common  to 
GISs.  Depending  on  the  sensor  in  use  at  any  one  time,  features  of  different  sues  will  be  of  interest.  It 
would  therefore  be  advantageous  to  have  features  of  similar  size  clustered  within  the  database,  once  again  for 
the  purpose  of  speeding  the  search  process.  This  size-related  clustering  would  be  subordinate  to  the 
geographical  clustering  described  above.  In  such  a  system,  one  or  several  disk  accesses  would  provide  all  the 
features  of  comparable  extent  within  a  given  area. 

(b)  Attribute  Search 

The  attributes  associated  with  the  features  stored  in  the  database  will  be  used  in  two  basic  ways.  First,  they 
serve  as  keys  in  the  initial  search  process.  The  control  generator  will  initially  (and  perhaps  subsequently) 
search  for  features  with  certain  attributes  within  a  specified  region.  Thus,  the  stored  feature  attributes 
become  the  primary  filter  for  choosing  candidate  ground  control.  Other  attributes  may  be  used  later  for 
other  purposes.  These  include  analyzing  the  quality  of  the  generated  ground  control  and  evaluating  the  limits 
of  error  in  the  preliminary  position  information  sent  to  the  image  matching  software. 

There  remains  some  question  about  the  best  place  to  store  this  attribute  data.  It  could  be  stored  in  a  separate 
database  existing  along-side  (and  keyed  to)  the  GIS.  This  would  minimize  the  size  of  the  GIS,  and  optimize 
the  purely  geometric  search  aspects  of  the  problem.  However,  it  appears  that  most  queries  will  involve  a 
combination  of  geometric  and  attribute  considerations.  Thus,  two  distinct  searches  of  the  separate  databases 
would  typically  be  required  for  each  query. 

4.1.1  Spatial  Database 

A  thorough  study  of  existing  and  theoretical  methods  for  the  representation,  storage  and  application  of  spatial 
information  was  an  important  component  of  the  Phase  I  work 


The  great  emphasis  that  the  generation  of  ground  control  places  on  lineal  features  (and  lineal  boundaries  of 
areal  features)  implies  that  data  representations  that  are  optimized  for  manipulating  point  and  lineal  data  as 
opposed  to  true  areal  representations  are  advantageous.  It  is  for  this  reason  that  a  number  of  areal 
representation  schemes  (such  as  quadtrees  and  (heir  derivatives)  are  not  optimal  for  this  work.  In  those  cases 
where  areas  of  imagery  are  important  to  matching  (as  in  actual  image  segments  or  in  using  windows  front 
simulated  imagery),  there  seems  no  need  to  maintain  a  representation  of  the  image,  because  it  is  the  image 
itself  that  is  relevant. 

The  specific  requirements  of  the  control  generator  for  memory  space  conservation,  feature  clustering 
(geographic  and  attribute),  rapid  access  and  accurate  database  updating  lead  us  to  prefer  a  highly  structured 
arc/node  format.  With  such  a  format,  points  are  represented  by  retaining  their  coordinates  with  respect  to 
some  reference  frame.  Lineal  features  are  represented  by  a  series  of  line  segments,  each  of  which  is  defined 
by  the  coordinates  of  its  endpoints.  Finally,  the  boundaries  of  areal  features  are  represented  as  closed 
polygons,  each  side  of  which  is  a  line  segment  defined  as  fot  lineal  features. 

The  greatest  advantage  the  use  of  a  point/vector  (or  arc/node)  format  affords  for  map  database  management 
and  ground  control  generation  is  the  logical  similarity  between  this  representation  and  the  likely  ground 
control  features  present  in  maps  and  images.  Ultimately,  all  registration  processes  rely  on  the  identification 
of  homologue  points.  Often,  particularly  in  dealing  with  large-scale  imagery,  these  points  are  derived  from 
the  more  visible  lineal  or  areal  features  in  the  image  (e.g.  intersections  of  lineal  features  or  centroids  of  areal 
features).  Existing  (and  probably  future)  digital  representations  of  map  information  are  composed  of  just 
Such  collections  of  points,  lines  and  polygons  as  are  used  in  arc/node-based  GISs  (e  g.  DFAD) 

The  highly  structured  arc/node  format  will  permit  the  GIS  module  to  efficiently  maintain  three  levels  of  data 
organization:  spatial,  topological  and  cartographic.  It  is  at  the  cartographic  level  that  each  feature  becomes 
more  than  simply  a  shape  and  location.  The  attributes  that  classify  and  describe  a  feature  arc  part  of  the  data 
object  (representing  (hat  feature)  manipulated  at  this  logical  level.  It  is  at  the  cartographic  level,  with  features 
as  shape  plus  characteristics,  (hat  most  human  interaction  with  (he  data  base  will  occur 

Operations  that  involve  the  geometric  relationships  between  features  will  operate  on  the  data  at  the  topological 
level.  The  union,  intersection,  tests  for  inclusion,  etc  of  features  can  be  done  here  without  attention  to  their 
exaci  location,  by  applying  topological  rather  than  coordinate  analysis  The  physical  extent  of  features  is 
represented  by  nodes,  edges,  and  faces  at  this  level  These  topological  ^presentations  are  created  and 
updated  from  the  cartographic  data  as  a  feature  is  added  to  the  database  The  many  to  many  relationships 
inherent  in  geographic  data  (e  g.  a  road  may  cross  many  land  areas,  and  a  land  atea  may  contain  main 
roads)  is  accommodated  by  links  (pointers)  between  nodes,  edges  and  laces  at  the  topological  level 
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The  combination  of  a  feature's  cartograpliic  and  topological  representations  is  then  geographically  clustered. 
At  this  'spatial'  level  of  data  organization,  features  are  placed  into  physical  storage  (disk)  with  other  features 
that  are  located  in  the  same  geographic  area.  This  is  important  in  order  to  speed  searches  for  features  within 
a  given  area.  Additional  data  organization  is  done  to  put  features  of  similar  size  near’  each  other  in  physical 
memory.  This  is  relevant  to  the  Smart  MCAG  problem  because  sensors  of  different  types  at  different  ranges 
will  produce  images  of  differing  resolution  The  data  base  will  contain  features  visible  from  satellites,  as  well 
as  small  objects  visible  only  from  low  altitudes.  Clearly,  the  registration  of  no  single  image  is  likely  to 
require  both  sorts  of  features. 

A  specific  example  of  a  spatial  database  that  appears  suited  to  further  development  of  of  automatic  control 
generation  is  the  Kork  Geographical  Information  System  (KGIS).  This  system  allows  for  the  clustering  of 
features  in  physical  memory  as  described  above  to  increase  the  search  performance  of  the  database.  It  is  an 
object  oriented  database,  so  that  all  information  about  the  nature  of  each  feature  and  the  operations  associated 
with  that  information  are  independent  of  the  software  external  to  the  database  itself.  This  permits  the 
flexibility  necessary  for  the  alteration  and  experimentation  that  will  be  an  important  part  of  subsequent  (Phase 
II)  work  on  the  Smart  MC&G  Control  Generator. 

The  database  structure  we  suggest  for  implementation  of  the  Smart  MC&G  Control  Generator  is  rather 
general.  It  reflects  current  work  in  spatial  data  handling,  and  requires  little  modification  to  be  entirely  well- 
suited  to  the  image  registration  application  we  are  discussing  here.  This  is  not  surprising,  as  the  data 

handling  aspect  of  the  problem  is  not  particularly  unusual  or  specific  to  control  generation.  The  more 

specific  aspects  of  the  problem  are  addressed  by  the  query  generator  that  actually  exploits  the  database,  and 
the  DEM  software  that  must  be  integrated  into  it. 

The  integration  of  digital  elevation  data  and  modeling  capabilities  into  the  spatial  database  will  be  a  significant 
project.  This  work  is  currently  being  performed  at  VEXCFL  in  an  effort  separate  from  the  SBIR  being 

reported  on  here.  Our  approach  will  be  to  provide  each  point  of  the  terrain  model  in  the  database  (KGIS)  an 

elevation  (z)  coordinate  as  well  as  the  planimetric  (x  and  y)  coordinates  The  elevation  of  features  not 
originally  a  part  of  the  DEM  can  be  (in  some  cases)  interpolated  from  the  grid  of  points  that  constitutes  the 
DEM  itself 

The  major  goal  of  all  phases  of  the  development  of  a  Smart  MC&G  Control  Generator  is  the  discovery  of  data 
and  processing  techniques  optimal  for  specific  automated  registration  tasks  Quantitative  modeling  of  the 
registration  process  in  order  to  evaluate  these  control  methods  and  features  will  lie  the  basic  tool  by  which  the 
optimal  combinations  are  found  The  control  system  must  lie  general  enough  to  accommodate  future 
advances  in  spatial  database  management  (both  in  haidware  and  software),  just  as  it  must  accommodate  future 
image  matching  and  feature  extraction  techniques  for  this  reason,  the  exact  n.ituie  of  the  database  svstem 
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used  for  current  work  must  be  divorced  from  the  control  generator  itself.  The  control  generation  system  will 
present  requests  for  feature  or  other  data  to  the  database  management  system.  The  exact  format  of  the  data 
within  the  database  should  not  affect  the  control  generator's  choice  of  the  data  to  be  used  for  registration. 
Our  conclusions  about  optimal  spatial  database  design,  and  our  choice  of  a  specific  existing  system,  are 
aimed  at  providing  the  best  currently  available  foundation  for  immediate  progress  in  quantitatively  evaluating 
the  components  of  the  registration  modeling  process. 

4.1.2  Image  Segments 

Small  portions  of  images  that  contain  examples  of  ground  control  features  can  be  retained  in  the  database 
system  in  order  to  facilitate  later  matching  processes.  These  image  segments  can  be  useful  for  rapid 
registration  in  certain  situations.  For  example,  a  well-defined  feature  present  in  an  area  that  is  periodically 
imaged  with  constant  sensor  type  and  geometry  can  in  itself  provide  the  needed  control.  In  this  case,  simple 
areal  correlation  of  an  image  segment  containing  the  permanent  feature  with  the  appropriate  region  in  the 
new  image  would  be  a  rapid  method  of  registration. 

Another  possible  use  of  image  segments  arises  in  registering  images  that  are  part  of  a  rapidly  sampled  time 
series  (e.g.  television-rate  infrared  images).  An  interest  operator  could  be  used  to  find  and  extract  windows 
containing  likely  control  features  in  the  first  of  a  series.  Subsequent  images  in  the  series  could  then  be 
registered  by  correlation  with  the  windows  extracted  from  earlier  images. 

The  use  of  image  segments  for  registration  in  these  rather  special  cases  would  be  managed  by  the  control 
generator's  query  generation  and  control  software.  A  library  of  image  segments  would  be  keyed  to  the 
corresponding  features  (where  appropriate)  in  the  spatial  database. 

4.2  Digital  Elevation  Modeling 

The  three-dimensional  nature  of  the  land  surface  to  be  represented  in  the  GIS  requires  that  elevation 
information  be  considered  when  generating  ground  control.  Furthermore,  the  use  of  a  variety  of  sensors, 
producing  images  with  differing  resolution  and  geometry,  implies  that  the  capability  for  rather  detailed 
modeling  of  the  terrain  be  an  integral  part  of  the  control  generating  system.  VEXCEL  is  currently  engaged 
in  development  work  to  integrate  elevation  coordinate  data  and  surface  modeling  capabilities  into  the  Kork 
Geographical  Information  System. 

One  of  the  primary  uses  of  a  digital  elevation  model  will  be  to  calculate  those  regions  of  the  area  in  question 
actually  visible  to  the  sensor  Registration  of  imagery  fiont  directly  oveihead  will  of  course  requite  little  or  no 
consideration  of  this  shadowing  effect  On  the  other  hand,  side-looking  imagery,  particularly  from  low 


altitudes,  must  interact  strongly  with  the  DEM  component  of  the  system.  A  second  major  use  will  be  the 
creation  of  simulated  images. 

This  software  module  will  have  the  following  capabilities: 

*  extract  elevation  data  from  GIS 

*  create  digital  elevation  model  of  the  necessary  precision  (may  vary  with  respect  to  specific 
registration  task) 

*  store/ main  tain  this  model  in  the  GIS 

*  shadow  analysis  (perspective  view)  of  region 

Because  of  the  computational  expense  involved  in  calculating  a  perspective  view  from  a  given  DEM  and 
sensor  position,  it  may  be  applied  only  judiciously.  In  a  dynamic  processing  setting  one  may  "update"  a 
previous  DEM  operation  by  adding  relevant  new  DEM  data,  avoiding  the  need  to  re-compute  the  entire 
model.  The  control  generation  system  must  determine  the  need  (and  required  accuracy)  for  consideration  of 
elevation  information  based  on  the  sensor  in  question  and  the  ground  relief  of  the  terrain  imaged. 

4.2.1  Simulated  SAR  Imagery 

The  digital  elevation  model  of  a  region  will  also  be  used  to  aid  in  creating  a  simulated  radar  image. 
Consideration  of  the  sensor  range,  resolution,  look  angle,  and  platform  heading  in  conjunction  with  the  DEM 
and  appropriate  introduced  noise  produces  an  pseudo-image  that  can  then  be  registered  with  the  actual  image. 
Since  the  simulated  image  is  produced  from  a  map-coordinate-based  DEM,  this  provides  geodetic  control  for 
the  actual  image.  Recent  work  reported  by  Guindon  and  Maruyama  indicates  that  this  technique  produces 
automatic  registration  of  SAR  imagery  with  accuracies  comparable  to  those  achieved  manually  (Guindon  and 
Maruyama.  1986]. 

The  simulation  procedure  used  assumes  that  the  radar  backscatter  (and  thus,  pixel  intensity)  is  dependent 
only  on  the  incidence  angle.  The  results  of  their  experimental  work  are  shown  in  Table  I.  The  greatest 
consistency  of  the  automated  procedure  with  manual  registration  is  found  when  large  (64x64  pixel)  windows, 
termed  registration  control  points  (RCPs).  are  extracted  from  the  actual  scene  for  comparison  with  the 
simulated  image.  Further,  the  use  of  adaptive  filtering  to  reduce  speckle  also  improves  the  registration. 

Poor  matches  (i.e.  low  peak  correlation  values  or  failures  to  find  a  correlation  peak  within  the  interior  of  the 
R CP)  tend  to  be  associated  with  image  areas  that  exhibit  a  small  radiometric  dynamic  range.  The  selection  of 
candidate  control  windows  on  the  basis  of  observed  gray-value  variance  would  eliminate  the  need  to  calculate 
correlation  values  for  these  unlikely  areas,  thus  increasing  the  efficiency  of  the  registration  process 
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TABLE  I 


This  table  illustrates  the  use  of  simulated  imagery  for  image  registration.  A  simulated 
synthetic  aperture  radar  image  is  created  by  using  a  digital  elevation  model.  The  pixel  gray- 
values  of  the  simulated  image  are  determined  by  the  incidence  angle  derived  from  an 
assumed  sensor  location  and  the  DEM.  Windows  from  the  simulated  and  actual  image  arc 
correlated  to  determine  registration  offsets.  Because  the  simulated  image  is  derived  from  a 
map-coordinate-based  DEM,  matching  of  the  two  images  produces  geodetic  control  for  the 
actual  radar  image. 

As  might  be  expected,  using  larger  windows  for  the  correlation  produces  more  accurate 
registration.  Additionally,  the  use  of  filtering  to  reduce  the  effects  of  speckle  improves  the 
matching.  Failures  (inability  to  find  a  correlation  peak)  tend  to  appear  in  windows  with  little 
radiometric  dynamic  range.  Thus,  selection  of  windows  based  on  observed  gray-value 
variance  should  improve  the  entire  process,  (from  Guindon  and  Maruyama,  1986) 
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4.2.2  Use  uf  SAR  Shadows 


Another  aspect  of  the  generation  of  simulated  SAR  images  from  a  DEM  is  the  possibility  of  using  radar 
shadows  as  features  for  image  registration.  The  sequence  of  images  below  (Figures  la.  lb.  2  and  3) 
illustrate  the  creation  of  shadow  ‘features'  in  a  simulated  SAR  image  [Donxik.  1985].  Figure  1  shows  the 
digital  elevation  model  used,  while  Figure  2  indicates  the  portions  of  the  scene  illuminated  given  the  sensor 
position  used.  In  Figure  3  the  simulated  image  is  shown.  Notice  that  features  generated  by  the  topography 
of  the  scene  include  bright  areas  due  to  radar  lay-over  as  well  as  the  shadow  regions.  These  types  of  features 
will  be  more  common  in  imagery  sensed  from  a  high  look  angle,  and  less  important  in  images  of  regions 
more  directly  beneath  the  sensor. 

4.3  Query  Generator  and  System  Control 

The  query  generator  is  the  component  of  the  system  that  exploits  the  map  database  for  the  purpose  of 
discovering  likely  ground  control  features.  Knowledge  of  the  characteristics  of  the  sensors  and  image 
matching  processes  is  utilized  to  guide  the  search  of  the  database  for  ground  control  relevant  to  the  current 
task.  The  choice  of  attribute  data  maintained  by  the  Geographical  Information  System  (GIS)  will  be  driven 
by  the  specific  character  of  the  feature  detection  and  image  matching  software,  as  well  as  by  the  exact  nature 
of  the  sensing  systems  used.  Choosing  a  GIS  structure  and  programming  techniques  that  emphasize  code 
maintainability  and  reliability  (i.e.  structured,  object-oriented  code)  will  allow  changing  techniques  in  either 
of  these  areas  to  be  easily  accommodated. 

In  addition  to  accessing  the  database  itself,  this  module  will  also  perform  overall  system  control  functions. 
This  includes  such  things  as  determining  the  need  for  DEM  or  simulated  image  construction,  and  the  choice 
of  rule  and  meta-rule  subsets  appropriate  to  specific  image  types  and  registration  tasks. 

The  use  of  expert  systems  for  maintaining  and  exploiting  spatial  databases  was  examined  during  Phase  I.  For 
the  near  future,  the  application  of  expert  system  techniques  to  Geographical  Information  Systems  will  be 
limited  to  small,  precisely  defined  problem  domains  [Robinson,  et  al.  1986].  Fortunately,  the  evaluation  of  a 
digital  map  database  for  candidate  ground  control  appears  to  be  one  such  precisely  defined  domain. 

This  query  generator  will  be  a  rule-based  expert  system,  designed  to  automate  the  choices  a  human  being 
would  make  if  tasked  with  the  problem  of  choosing  initial  ground  control  for  differing  image  types.  It 
communicates  with  the  data  itself  only  through  the  storage  and  retrieval  protocols  intrinsic  to  the  GIS  system. 
Thus,  we  avoid  building  dependence  on  each  other  into  either  module.  Modification  of  the  rules  and  meta¬ 
rules  themselves  will  be  necessary  as  new  techniques  (and  new  feature  types)  are  developed. 


FIGURE  I 


The  digital  elevation  model  used  for  an  experiment  with  producing  radar  shadows  in  SAR 
imagery.  Figure  la  (above)  shows  the  model  as  seen  from  above.  The  lighter  color 
indicates  greater  elevation.  Figure  lb  (below)  provides  a  schematic  of  the  model  as  seen 
from  the  side.  The  elevations  in  the  model  range  from  zero  to  four  kilometers,  while  the 
sensor  is  placed  at  a  height  of  I  I  kilometers.  The  numbers  (0.1.2)  indicate  regions  in  the 
slant  range  image  in  which  a  shadow,  regularly  illuminated  region,  and  region  of  lay-over, 
respectively,  would  appear,  (from  Domik.  1985) 
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At  its  most  primitive  level,  this  rule-based  approach  will  allow1  the  implementation  of  otir  understanding  of 
map-guided  ground  control  for  a  given  sensor  type.  That  is.  as  decisions  are  made  concerning  the  utility  of 
certain  feature  types  or  attributes,  the  query  generator  will  be  modified  to  reflect  the  importance  to  be  given 
that  feature  type  or  attribute.  Ultimately,  however,  the  usefulness  of  the  query  generator  will  extend  beyond 
this:  comparisons  of  the  relative  utility  of  differing  sensor  types  and  image  matching  techniques  in  pursuit  of 
various  registration  tasks  will  be  efficient  and  accurate.  Decisions  concerning  these  relative  strengths  and 
weaknesses  can  then  be  incorporated  into  the  rule  set. 

Three  basic  concepts  of  data  representation  and  software  implementation  are  important  to  the  Smart  Control 
Generator.  The  geographical  data  itself  is  represented  in  the  previously  discussed  arc/node  structure.  This 
data  type  should  not  be  confused  with  the  object-oriented  approach  recommended  for  implementation  of  the 
database  management  system.  The  objects  manipulated  by  the  database  may  include  geographical  data 
represented  in  an  arc/node  data  structure,  but  may  also  include  attribute  data  and  object  processing  routines 
that  are  not  represented  with  such  a  structure.  Finally,  the  query  generation  system  that  intelligently  applies 
the  functions  of  the  database  manager  will  encode  the  knowledge  useful  for  control  generation  as  a  set  of 
rules  and  meta-rules  to  direct  the  manipulation  of  the  various  objects  maintained  in  the  database. 

4.3.1  Overview  of  Expertise  for  Control  Information 

The  task  of  registering  imagery  with  map  data  is  an  important  problem.  However,  images  and  maps  are 
quite  different  data  formats.  Images  are  sensor  recordings  of  the  received  signatures  of  scatters  of  some  type 
of  electromagnetic  radiation,  whereas  a  map  is  some  two-dimensional  compilation  of  elevation  and  feature 
data  by  location.  Maps  are  ususally  at  a  very  different  scale  than  images,  and  are  also  much  more  sparse  in 
detail.  Consequently,  registration  cannot  be  performed  on  the  basis  of  some  local  gray-value  similarity 
corresponding  to  local  scene  similarity,  as  in  image-to-image  registration.  Instead,  registration  can  be 
attempted  in  two  distinct  ways.  One  way  is  by  direct  map-to-image  matching.  Another  way  is  to  produce  a 
synthetic  image  from  the  map  data  that  corresponds  to  some  projected  view  similar  to  that  of  the  real  image, 
and  then  performing  a  registration  between  the  real  and  synthetic  images. 

Direct  map-to-image  registration  is  performed  on  the  basis  of  finding  a  correspondence  between  some 
identifiable  map  feature  itself  and  its  signature  within  the  image.  Such  features  are  called  control  information. 
Such  information  is  ideally  in  the  form  of  distinguishable  points,  but  often  is  in  the  form  of  identifiable  line 
segments  and  contours,  either  open  or  closed.  Because  synthetic  images  contain  less  microstructure  than  real 
images,  any  matching  between  the  real  and  synthetic  image  must  proceed  on  the  basis  of  the  real  and 
simulated  signatures  of  map-recorded  features.  Therefore,  control  points  are  also  important  for  the  latter  type 
of  registration. 


In  order  lo  attempt  an  automation  of  this  type  of  registration,  there  must  exist  an  automated  process  of 
choosing  control  information.  Given  some  desired  accuracy  of  registration,  such  an  automated  process  must 
decide  which  features  to  select,  how  many,  and  with  what  spatial  distribution.  Examining  the  question  of  the 
feasibility  of  automating  expertise  for  the  selection  of  control  information  is  the  subject  of  this  section. 

The  technology  of  expert  systems  is  currently  receiving  a  great  deal  of  attention.  Currently. however,  there 
does  not  exist  a  large  number  of  working  expert  systems  which  contain  large  knowledge  bases  containing 
hundreds  or  thousands  of  rules.  This  is  because  the  process  of  distilling  domain-specific  expertise  into  a 
suitable  format  for  automatic  processing,  such  as  rules,  is  tedious,  time-consuming,  and  not  straight-forward. 
Another  reason  is  that  not  every  subject  domain  is  amenable  to  the  current  state  of  expert  system  technology. 

Some  good  "rules  of  thumb"  concerning  the  applicability  of  expert  system  technology  to  a  domain  are: 

*  A  typical  problem  can  be  solved  by  an  expert  in  a  few  minutes  to  a  few  hours,  ie.  the  domain  is 
reasonably  bounded. 

*  The  expertise  is  decomposable  into  modules  of  expertise. 

*  The  expertise  can  be  taught  to  a  novice,  ie.  it  is  succinctly  describable. 

We  believe  that  the  problem  of  finding  good  control  information  satisfies  these  criteria.  However,  satisfying 
such  "rules  of  thumb"  does  not  constitute  a  demonstration  of  feasibility.  Clearly,  what  is  required  for  every 
separate  case  is  a  more  detailed  analysis  of  the  structure  of  the  appropriate  working  knowledge. 

Another  issue  concerns  whether  the  implementation  of  the  domain-specific  knowledge  should  procedural,  ie. 
they  do  not  separate  the  logical  from  the  control  content  of  the  program.  The  control  structure  determines  the 
order  of  execution  in  a  program.  In  procedural  programs,  this  order  of  execution  is  intimately  connected  with 
the  order  of  its  statements.  However,  programs  also  have  a  logical  content  which  is  determined  by  the  logical 
relationships  among  its  logical  clauses. 

In  non-procedural  programming,  these  logical  relationships  do  not  depend  on  physical  relationships  among  a 
program’s  statements,  such  as  their  order.  The  control  and  logical  contents  are  separated,  and  so  what  is  to 
be  done  can  be  separated  from  how  something  is  to  be  done.  This  separation  of  functions  is  very  convenient 
for  implementing  domain-specific  knowledge.  Individual  statements  representing  such  knowledge  can  be 
added  or  removed  from  the  knowledge  base  without  regard  to  their  order.  Thus  modification  of  the  knowledge 
base  can  proceed  entirely  on  the  basis  of  its  logical  content.  PROLOG  is  an  example  of  a  language 
supporting  such  logic  programming.  In  PROLOG  a  knowledge  base  is  implemented  using  "rules".  These 
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rules  take  the  familiar  form  of  an  "If  (hypothesis)... then  (conclusion)"  paradigm.  This  type  of  rule-based 
programming  will  be  the  choice  for  implementing  the  expertise  for  selecting  control  information. 

The  knowledge  base  will  take  the  form  of  rules  used  by  experts  to  choose  control  information.  These  rules 
are  generally  based  on  heuristics,  and  "rules  of  thumb"  commonly  employed  by  practicing  experts.  The  value 
of  a  rule  should  be  its  applicability  in  a  wide  variety  of  typical  cases. 

Rules  with  negative  conclusions  are  important  also.  For  example,  a  rule  which  cautions  against  picking 
mountain  peaks  as  control  points  in  direct  map-to-image  matching  is  as  useful  as  one  which  positively 
chooses  the  intersection  of  two  roads  on  flat  terrain.  Such  reasoning  allows  the  system  to  eliminate 
possibilities. 

There  will  likely  be  a  type  of  "gray  area"  between  positive  and  negative  rules  wherein  it  is  not  clear  that  a 
feature  is  suitable  or  not.  This  means  that  rules  which  eliminate  a  candidate  feature  are  not  necessarily 
converses  of  rules  that  choose  a  candidate.  Therefore,  most  rules  will  not  give  both  necessary  and  sufficient 
conditions  for  choosing  a  particular  feature  as  candidate  control  information.  Such  a  situation  creates  the 
possibility  of  inconsistent  rules  being  added  to  the  knowledge  base.  However,  it  is  hoped  that  usually  rules 
will  be  related  by  being  partial  converses  of  each  other.  Therefore,  inconsistencies  should  be  relatively  rare. 

4.3.2  Preliminary  Investigation  of  Control  Information  Expertise 

Expertise  for  the  problem  domain  of  iinage-to-map  mode  of  matching  proceeds  by  registering  the  real  image 
directly  to  the  map  or  indirectly  using  a  synthetic  image.  However,  regardless  of  the  sensor  type  employed 
and  desired  mode  of  registration,  the  expertise  must  answer  the  questions: 

*  Which  map  features  to  choose? 

-  man  made 

-  natural 

*  How  many  features  should  be  chosen  and  what  should  be  their  distribution? 

The  answers  to  the  question  of  how  many  features  are  required  involve  what  type  of  matching  transformation 
is  to  be  estimated.  The  question  of  the  distribution  of  the  features  involves  avoidance  of  singularities  and 
sensitivity  of  the  transformation  equations  to  the  relative  positions  of  the  features. 

The  answers  to  the  question  of  which  features  to  choose  depend  on  the  type  of  imaging  sensor  type  and  the 
mode  of  registration.  However,  for  each  such  combination  of  sensor  type  and  mode  of  registration,  there  are 
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a  number  of  characterisiics  of  map  features  that  determine  the  suitability  of  a  feature  for  achieving  good 
control  information.  We  identify  some  of  these  characteristics  below: 

*  strength  of  return 

-  sensor  wavelength 

-  sensor  spatial  resolution 

-  feature  geometry 


-  shape 

-  feature  material  properties 

-  weather-induced  moisture 

-  illumination  power  onto  scene 

-  illumination  direction 


*  magnitude  of  local  contrast 

-  expected  signal  strength  difference  between  feature  and 

local  background 

-  sensor  spatial  resolution 

-  sensor  radiometric  resolution 

*  geometrically  distinctive  components 

-  points 

-  contours 

-  open 

-  closed 

*  potential  for  confusion 

-  local  similarity 

-  geometrically  induced  artifacts  and  distortions 

As  an  example,  consider  the  case  of  direct  map-to-image  matching  for  SAR.  The  scale  of  most  man-made 
features,  other  than  entire  cities,  is  such  that  they  are  important  to  registration  only  if  higher  resolutions  are 
employed.  Often,  such  objects  give  rise  to  point  or  lineal  features  which  are  suitable  for  registration.  Cities 
often  provide  strong  lineal  returns  if  some  of  their  streets  run  perpendicular  to  the  direction  of  illumination. 
Road  and  seismic  line  crossings  may  provide  good  point  returns. 
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However,  many  such  man-made  objects  involve  height  discontinuities  with  their  surroundings,  such  as 
buildings  or  lowers.  Therefore,  care  must  he  taken  to  account  for  height-related  imaging  anomalies  that  are 
peculiar  to  radar  imagery,  such  as  lay-over,  radar  shadows,  and  edge-migration.  Imaging  of  extended 
regular  areas  such  as  cities  may  involve  confusion  as  to  which  streets  are  actually  being  imaged. 

An  example  of  a  type  of  natural  feature  providing  good  point  returns  is  a  small  river  mouth  in  a  flat  area.  An 
extended  crossing  that  is  wider  can  still  used,  but  now  some  point-like  derived  feature  such  its  cencoid  is 
often  used.  River  windings  may  also  be  useful  as  contour-like  features.  However,  the  surrounding  terrain 
must  be  relatively  flat  or  else  geometric  distortion  will  be  present,  resulting  in  improper  location  of  the 
feature  in  the  image.  This  concern  with  topography  makes  it  difficult  to  use  mountainous  terrain,  drainage, 
or  peaks  for  control  points.  On  the  other  hand,  if  synthetic  images  are  used  then  such  terrain  relief  can  be 
used  for  registration.  This  is  because  the  geometric  distortions  inherent  in  SAR  imagery  of  such  scenes  will 
be  faithfully  reproduced  in  the  corresponding  synthetic  images. 

An  initial  task  for  the  query  generator  will  be  to  determine  whether  topographic  shadowing  is  significant  to 
the  current  registration  goal.  This  will  involve  consideration  of  the  sensor  geometry  and  type,  as  well  as  the 
general  degree  of  ground  relief  in  the  region  of  interest.  If  needed,  the  query  generator  must  then  calculate 
those  areas  of  the  region  actually  visible  to  the  sensor.  Only  those  areas  will  be  subject  to  subsequent 
investigation. 

Clearly,  the  type  of  sensor  used  to  create  the  image  to  be  registered  is  the  most  basic  parameter  into  the 
query  generator.  Tile  sensor  type  will  influence  the  sizes,  types,  and  attributes  of  the  features  to  be  searched 
for  in  the  database.  In  formulating  the  initial  query,  the  generator  must  also  account  for  some  other  aspects 
of  the  particular  task  in  work.  Environmental  factors  such  as  temperature  and  precipitation  will  affect  both 
the  sensor  performance  and  the  feature  characteristics.  The  degree  of  registration  precision,  and  the  speed 
with  which  that  registration  is  performed  will  also  influence  the  choice  of  ground  control  features. 

Typically,  choices  of  control  features  will  be  made  as  a  result  of  a  number  of  tests  concerning  the  specific 
registration  task,  the  image  matching  and  registration  techniques  available,  and  the  feature  content  of  the 
region  in  question.  Several  qualitative  examples  will  serve  to  illustrate  the  formulation  of  queries  to  the 
database. 

1)  If  ((image-type  *  aerial-SAR)  and 
(topographic-relief  =  'low')) 

then  (find  (search-area  =  defined  by  initial  sensor  geometry)  and 
(feature-type  =  road)  and 
(orientation  =  'near-parallel-to-flight-path')) 


In  this  case,  a  feature  of  a  specific  type  is  being  extracted:  the  knowledge  that  roads  have  surface 
material  compositions  and  shapes/geometries  (e  g  long  and  thin,  relatively  flat)  that  are  good  for  aerial 
SAR  control  is  not  explicitly  expressed.  The  possibility  that  roads  may  be  mis-located  in  SAR  imagery 
of  areas  with  significant  relief  is  accounted  for  in  the  second  test.  The  geographical  limits  of  the  search 
area  may  be  determined  by  any  of  the  dead-reckoning  techniques  that  provide  preliminary  sensor 
location  and  orientation  information.  In  this  particular  case,  the  search  area  determination  would  not 
require  consideration  of  DEM  information,  because  the  topographic  relief  has  already  been  found  to  be 
low.  Finally,  the  orientation  of  the  road  with  respect  to  the  flight  path  must  be  appropriate.  An  example 
of  the  importance  of  this  requirement  to  radar  registration  can  be  seen  in  Figures  4a  and  4b  [Domik. 
1986].  The  actual  evaluation  of  most  if  not  all  attributes  such  as  orientation  =*  near-parallel-to-flight- 
path’  will  be  quantitative:  the  determination  of  appropriate  metrics  (and  values  appropriate  for  the  control 
generation  task)  for  such  characteristics  being  a  significant  portion  of  subsequent  effort  in  this  area. 

2)  If  ((image-type  =  SIRB-SAR)  and 

(topographic-relief  =  high')) 
then  ((create-siniulated-image 

with(region  =  determined  by  initial  sensor  geometry))  and 
(extract  windows  from  simulated  image  with  large  radiometric 
dynamic  range)) 

This  meta-rule  finds  windows  suitable  for  the  automated  SAR  registration  technique  described  earlier 
[Guindon  and  Maruyama.  1986].  It  would  make  use  of  the  DEM  data  for  the  image  simulation  process. 
More  sophisticated  simulation  methods  which  would  also  require  knowledge  of  the  surface  character  and 
feature  content  can  be  envisioned. 

3)  If  (image-type  =  aerial-photo) 

then  ((calculate-relevant-feature-size 

with(sensor  position,  resolution))  and 
(find  (search-area  =  defined  by  initial  sensor  geometry)  and 
(feature-shape  =  linear-intersections)  and 
(feature-contrast  =  'bright'))) 

In  this  example,  the  control  information  is  searched  for  on  the  basis  of  its  characteristics  as  an  image 
feature.  It  is  not  identified  as  a  particular  type  (as  were  (he  'roads'  in  example  I).  Rather,  any 
intersection  of  'bright'  linear  features  may  be  useful  for  registering  aerial  photos,  and  so  all  such 
features  are  extracted.  The  size  (width)  of  features  useful  for  this  task  will  depend  on  the  resolution  and 
range  of  the  sensor:  this  illustrates  the  benefit  of  being  able  to  efficiently  search  the  spatial  database  for 
features  of  a  given  size  range,  as  well  as  features  located  within  a  given  region. 


The  qiierv/conirol  module  must  also  supers  ise  all  subsequent  searches  (o  the  01 S  This  would  be  necessan 
if  the  initial  search  did  not  find  enough  features,  or  if  the  image  matching  software  cottld  not  find  them  in  ilie 
image.  In  these  cases,  a  search  similar  to  the  first,  but  with  relaxed  conditions  could  be  executed  Alter  an 
initial  matching/registration  step,  registration  could  be  refined  by  using  features  not  initially  reported  For 
example,  a  feature  in  a  cluster  may  not  have  been  used  in  the  first  iteration  because  of  the  possibility  of  an 
incorrect  match  with  similar  objects  nearby.  The  initial  registration  can  be  used  to  restrict  the  subsequent 
search  space  so  that  this  feature  then  becomes  usable. 

4.4  Error  Estimation 

In  addition  to  the  features  themselves,  estimates  of  the  probable  limits  of  error  between  the  preliminary 
registration  and  actual  registration  must  also  be  passed  lo  the  image  matching  software.  These  estimates  are 
used  to  define  the  region  of  the  image  within  which  the  matching  software  will  search.  The  query/control 
mechanism  will  calculate  these  error  probabilities  based  on  the  accuracy  of  the  sensor  geometry,  the  sensor 
resolution,  and  the  character  of  the  feature  and  image  matching  processes. 

A  classical  concept  involves  the  computation  of  approximate  image  coordinates  for  a  specific  control  feature. 
An  error  envelope  is  defined  for  the  image  area  in  which  the  matching  image  feature  should  be  found  The 
matching  then  consists  of  finding  an  object  of  known  properties  in  the  search  area. 

The  evaluation  of  the  sources  of  error  in  registering  different  image  types  will  be  an  important  aspect  of 
subsequent  (Phase  II)  work.  The  errors  inherent  in  the  initial  sensor  position  determination  systems  (e  g. 
the  various  dead-reckoning  tools)  and  in  the  image  matching  techniques  will  be  an  important  consideration 
for  the  extraction  of  control  information. 

4.5  Performance 

A  major  concern  in  the  design  of  a  Smart  MC&G  Control  Generator  is  the  speed  at  which  images  can  be 
registered  to  a  common  reference  frame.  In  general,  the  goal  is  to  perform  such  operations  in  ’real-time' 
However,  precisely  what  is  meant  by  real-time  varies  with  the  image  type.  An  infrared  sensor  operating  at  a 
scan  rale  of  30  Hz  requires  that  each  new  frame  by  registered  in  something  less  than  0.033  sec,  while  the 
registration  of  photographs  of  an  area  taken  days  apart  is  under  virtually  no  time  constraint. 

Fortunately,  the  registration  of  frames  from  a  continuously  operating  scanner  (such  as  an  infrared  sensor) 
can  be  performed  rapidly.  Each  frame  can  be  registered  by  relying  on  the  regis.  aiion  of  the  previous  frame 
This  is  done  by  predicting  the  location  of  features  in  the  new  frame  based  on  the  motions  of  those  features  in 
past  frames.  Thus,  the  search  space  for  the  actual  image  matching  (for  refinement  of  the  registration)  is  kept 

quite  small. 


FIGURE  4 


This  figure  shows  the  dramatic  efTect  that  the  flight  orientation  of  an  SAR  sensor  can  have 
on  the  features  visible  in  the  image  it  produces.  These  two  SIR-B  images  arc  of  the  same 
region  in  Argentina  The  image  on  the  left  was  produced  as  the  sensor  was  moving  from 
the  upper-left  to  lower-right,  while  in  the  image  on  the  right  the  sensor  is  moving  from 
lower-left  to  upper-right. 

Notice  the  linear  features  that  are  strongly  visible  in  the  left  image.  These  are  streets  (and 
houses  along  the  streets)  that  are  aligned  nearly  parallel  to  the  llight-path.  The  same  streets 
are  not  visible  in  the  right  image,  because  they  are  nearly  perpendicular  to  the  flight  path, 
(from  Domik.  1986) 


Registration  of  synthetic  apctiure  radar  images  proceeds  on  a  somewhat  different  time-scale  The  intent  is  to 
register  imagers  as  it  is  received  at  (lie  platform  for  example,  assume  an  aircraft  ground  speed  of  500 
km. hour,  and  an  image  swath  of  20  km  in  width.  Then  a  20  km  by  20  km  image  is  received  every  144 
seconds  Generation  of  ground  control  and  image  matching  should  occur  in  something  less  than  than  144 
seconds  in  this  case 

An  important  aspect  of  future  work  in  this  area  will  lac  the  identification  of  those  aspects  of  the  control 
generation  and  image  registration  processes  that  can  be  implemented  in  parallel  and  pipelined  systems.  The 
growing  availability  of  such  hardware,  and  the  parallel  nature  of  many  image  processing  tasks,  makes  this 
sort  of  development  attractive  for  real-time  applications 

The  generation  of  candidate  ground  control  should  clearly  be  done  as  rapidly  as  possible,  as  the  actual  image 
processing  and  matching  processes  are  likely  to  be  more  time  consuming  than  a  database  search  However, 
the  generator  system  should  take  into  account  the  time  constraints  of  a  particular  registration  task  as  well  as 
the  accuracy  requirements.  This  will  allow  the  proper  balance  between  speed  and  precision  to  be  achieved. 


Section  5 


Related  Work 


The  earliest  work  in  the  field  of  automated  image  registration  concentrated  on  developing  algorithms  to 
perform  the  calculation  of  the  transformation  function  bv  which  the  coordinate  system  of  an  image  can  be 
related  to  another.  The  time-consuming  and  error-prone  calculation  of  an  affine  transformation  or  the 
Helmert  transformation  is  a  task  obviously  suited  to  even  the  earliest  digital  computer  systems.  However 
these  early  registration  aids  still  required  that  a  human  operator  delect  and  input  the  coordinates  of  distinct 
features  visible  in  both  images.  Automation  of  this  task  was  first  proposed  and  demonstrated  with  optical 
imagery  that  featured  little  if  any  geometric  or  radiometric  differences.  Extensions  to  dissimilar  imagery,  or 
to  the  registration  of  imagery  to  non-image  references,  remains  a  relatively  undeveloped  field  of  study. 

Many  of  the  various  aspects  of  the  problem  of  automated  image  registration  have  been  studied,  although  the 
integration  of  these  subtasks  into  a  working  system  remains  a  goal  yet  to  be  achieved.  Our  current  efforts  in 
developing  an  automated  control  generation  system  serve  not  only  to  closely  examine  that  particular  subtask, 
but  to  provide  a  framework  to  model  the  entire  registration  process.  Because  of  the  need  for  a  control 
generating  system  to  account  for  all  aspects  of  image  registration,  an  extensive  review  of  past  and  current 
work  in  image-to-image  matching,  spatial  data  representation  and  image-to-map  registration  was  performed 
prior  to  and  during  the  execution  of  Phase  I.  The  following  will  step  through  work  done  on  the  various 
component  problems  that  contribute  to  the  Smart  Control  domain. 

5.1  Image-to-lmage  Matching 

The  automated  detection  of  homologue  points  in  a  pair  of  images  touches  many  of  the  basic  areas  of  research 
in  image  processing.  One  early  (but  still  used)  approach  involves  the  direct  correlation  of  subsets  of  the  two 
images.  The  normalized  cross-correlation  technique  [Quam.1971]  finds  the  coordinates  at  which  one 
subimage  best  corresponds  to  another,  producing  a  single  ground  control  point.  This  correlation  can  also  be 
done  in  the  frequency  domain  through  the  use  of  Fast  Fourier  Transforms,  for  an  improvement  in  the  speed 
of  the  matching  [Anuta.  I970J.  Emmert  and  McGillem  [I97JJ  discover  only  those  coefficients  of  the  affine 
transformation  needed  to  account  for  geometric  distortion  between  images  in  the  Fourier  domain  and  then  use 
simple  image  segment  correlation  to  find  the  translation  components  of  the  transformation. 


A  class  of  algorithms  termed  Sequential  Similarity  Detection  Algorithms  (SSDAs)  has  been  presented  in  an 
attempt  to  decrease  the  computational  expense  of  image  matching  fRarnea  and  Silverman.  1972].  These 
algorithms  attempt  to  quickly  identify  olfset  values  for  which  the  correlation  coefficient  is  poor.  They 
randomly  sample  the  pixel  locations  in  the  two  images  rattier  than  systematically  examine  each  location.  A 
potential  match  is  discarded  when  enough  (randomly  chosen)  pixels  have  been  examined  to  indicate  that  the 
correlation  coefficient  for  this  offset  will  exceed  a  certain  threshold.  This  method  suffers  from  the  lack  of  a 
statistical  description  of  the  quality  of  the  match. 

Still  another  way  to  reduce  the  cost  of  correlation  is  to  reduce  the  number  of  possible  matches  to  be  tested 
with  a  coarse-to-fme  pyramid  structure  [Moravec.  1977],  Correlation  begins  between  coarse  images  that 
span  a  larger  portion  of  the  scene  than  the  finer  images.  Discovery  of  a  matching  offset  at  this  coarse 
resolution  reduces  the  size  of  the  search  space  for  matches  at  finer  resolutions.  These  multiple  resolution 
techniques  have  been  used  by  several  researchers:  reviews  of  some  of  this  work  have  been  prepared  by  Burt 
[1982;  1983].  The  success  of  this  approach  depends  on  the  'pull-in’  range  of  the  regions  being  correlated 
[Crombie.  1983].  A  correlation  that  peaks  sharply  only  very  near  a  good  match  is  said  to  have  a  narrow 
pull-in  range.  In  any  case,  it  is  advantageous  to  register  two  images  by  looking  for  matches  in  regions  that 
seem  likely  to  exhibit  high  correlation.  Various  interest  operators,  filters  that  isolate  features  that  stand  out 
from  their  surroundings,  have  been  proposed  to  discover  such  regions  [Moravec,  1977;  McConnell.  1987], 

Several  types  of  imagery  have  been  registered  by  discovering  offsets  between  windows  in  the  images  through 
area  correlation.  Anuta's  work  with  fast  Fourier  transforms  dealt  with  multispectral  scanner  and  vidicon 
imagery  from  the  Earth-Resources  Technology  Satellite  (ERTS)  [Anuta,  1970],  Images  from  the  second 
Shuttle  Imaging  Radar  mission  (SIR-B)  have  been  registered  by  correlating  windows  from  the  images  with 
segments  of  a  simulated  image  derived  from  topographic  information  about  the  area  [Guindon  and 
Maruyama,  1986]. 

Techniques  that  continue  to  use  some  version  of  image  correlation,  but  that  operate  with  features  detected  in 
the  images  have  also  been  explored.  For  example,  methods  that  attempt  correlation  using  detected  edges  in 
the  images  are  numerous,  and  all  rely  on  the  research  done  in  the  field  of  edge  detection  itself.  Classical 
work  in  this  area  includes  (he  development  of  an  variety  of  operators  for  detecting  edge  elements  (intensity 
contrasts)  in  images.  The  Roberts,  Sobel  and  Kirsch  operators  are  all  well-known  examples  [Roberts.  1965: 
Sohel,  1973:  Kirsch.  1971],  Another  set  of  operators  act  by  matching  image  data  to  pre-defincd  models  of 
ideal  edges  of  varying  shape.  The  Hueckel  operator  and  later  refinements  by  Nevada  are  of  this  type 
[Hueckel,  1971;  Nevatia,  1975],  The  Hough  transform  [Hough.  1962]  and  its  generalization  by  Duda  and 
Hart  [1972]  and  Ballard  [1981a]  detects  linear  features  of  arbitrary  shape  by  use  of  appropriately 
parameterized  transformations.  Image  matching  using  edges  usually  relies  on  the  correlation  of  entire  edge 
images.  An  early  approach  developed  for  Landsat  image-to-image  matching  [Nack.  1974]  has  been  used  and 
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expanded  by  oilier  aulhors.  The  matching  of  images  in  order  to  produce  stereo  (three-dimensional) 
information  from  image  pairs  has  also  been  performed  with  a  segment-based  approach  [Mcdioni  and  Nevada. 
I985J. 

An  edge  operator  that  is  currently  receiving  considerable  interest  in  the  image  processing  community  is  based 
on  research  into  the  way  edges  and  boundaries  are  recognized  by  h  ;mans.  Edge-based  image  matching 
algorithms  derived  from  such  studies  utilize  the  zero-crossings  of  the  Marr-Hildreth  edge  operator  (i.e.  the 
Laplacian  of  the  two-dimensional  gaussian  function)  [Marr  and  Poggio.  1976;  Marr  and  Hildreth,  1980]. 
An  algorithm  of  this  type  has  been  used  for  automating  stereopsis  of  aerial  photographs  [Grimson,  1985]. 
Another  promising  application  of  this  operator  has  the  goal  of  automatic  SAR  image-to-image  matching 
[McConnell.  1987],  This  work,  being  performed  at  VEXC12L  Corporation,  is  now  being  expanded  into  (he 
realm  of  SAR-to-optical  image  matching. 

Early  implementations  of  the  Marr-Hildreth  operator  were  quite  slow,  the  large  two-dimensional  masks  used 
in  convolving  the  gaussian  function  with  the  image  requiring  substantial  computation  time.  Recently, 
however,  a  much  more  efficient  method  has  been  presented  [Wiejak.  1985].  Wiejak  shows  that  Marr- 
Hildreth  masks  can  be  separated  (i.e.  factorized)  and  performed  as  a  sequence  of  one-dimensional 
convolutions. 

One  particular  measure  of  the  degree  of  correspondence  between  the  Marr-Hildreth  zero-crossings  of  two 
images  considers  the  distance  between  those  zero-crossings  at  a  given  offset.  These  distances  can  be  easily 
computed  by  transforming  the  zero-crossings  array  created  by  the  operator  into  an  array  of  numbers 
representing  the  distance  from  each  pixel  to  the  nearest  zero-crossing.  Such  an  array  is  created  with  only 
two  passes  through  the  zero-crossing  array  by  a  process  called  'chamfering'  [Barrow,  et  al,  1977]. 

The  vast  majority  of  image  matching  research  addresses  pairs  of  imagery  that  are  similar  and  present  only 
geometric  differences.  Synthetic  Aperture  Radar  (SAR)  does  not  fall  into  this  category.  Minor  differences  in 
sensor  position  can  drastically  affect  the  appearance  of  this  type  of  image.  Future  research  is  expected  to 
become  progressively  involved  with  matching  dissimilar  images. 

5.2  Imnge-to-Map  Registration 

A  major  source  of  map  data  is  a  variety  of  remote  sensing  imagery  including  aerial  or  space  photography, 
electro-optical  sensing,  synthetic  aperture  radar,  and  infrared  sensing.  Tiie  integration  of  this  data  into 
existing  databases  is  a  complex  problem,  and  most  of  the  processes  involved  are  currently  labor-intensive. 
The  first  step  is  clearly  to  register  the  new  image  with  the  existing  database.  Only  then  can  geometric  and/or 
attribute  information  extracted  from  the  image  be  added  to  or  compared  with  the  existing  data.  The  important 
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•asks  of  feature  detection,  change  detection  and  automated  database  updating  all  require  this  initial 

registration. 

This  registration  procedure  is  presently  accomplished  by  a  human  operator  who  selects  corresponding  points 
from  the  database  and  the  image  of  interest.  These  points  can  be  used  to  determine  the  sensor  position  and 
attitude  (exterior  orientation)  and  the  coordinate  transformation  needed  to  move  from  the  reference  frame  of 
the  image  to  that  of  the  database  and  vice  versa.  In  special  cases  the  points  may  also  serve  to  determine 
parameters  of  the  sensor's  internal  imaging  geometry  (inner  orientation).  The  calculation  of  the  exterior 
orientation  and  transformation  are  relatively  well-understood  and  tools  for  machine-support  exist.  The  time- 
consuming  aspect  of  the  registration  process  is  the  manual  identification  and  reporting  of  the  ground  control 
points.  The  creation  of  autonomous  systems  for  that  process  is  tints  of  great  value;  considerable  effort  needs 
to  be  spent  to  develop  such  systems. 

In  recent  years,  the  amount  and  quality  of  geographical  data  collected  by  aerial  and  satellite  sensing  systems 
has  greatly  increased.  The  numerous  commercial,  environmental  and  military  applications  of  such  data 
requires  that  much  more  efficient  techniques  for  the  maintenance  of  digital  map  databases  be  implemented. 
Thus,  the  development  of  a  "Smart  Mapping,  Charting  and  Geodesy  (MC&G)  Control  Generator"  is  a  centra! 


The  use  of  various  examples  of  spatial  data  storage  for  the  registration  of  images  has  been  a  major  aspect  of 
the  work  done  with  spatial  databases  in  general.  The  automated  generation  of  control  information  for  such 
registration  has  been  an  important  aspect  of  that  work,  and  thus  a  significant  base  of  previous  research  exists. 
Much  of  this  research  builds  upon  early  work  done  by  Barrow,  et  al  [1977].  in  what  they  termed  'map- 
guided'  feature  analysis. 

Linear  features  have  always  been  attractive  candidates  for  the  registration  of  images  to  maps,  and  work  by 
Tennenbaum,  et  al  [1978],  and  Fischler,  et  al  [1979],  has  concentrated  on  these  types  of  features.  One 
technique  suggested  by  these  workers  employs  a  distance  array  created  by  the  'chamfering'  procedure 
described  earlier.  This  array  then  guides  a  line-following  process  to  identify  linear  features  in  the  image. 

The  MAPS  system  developed  at  Carnegie-Mellon  University  relates  a  map  database  to  an  image,  but  does  not 
specifically  address  the  automated  generation  of  control  information  [McKeown.  1982;  McKeown  and 
Denlinger.  1984;  McKeown,  et  al.  1984],  This  work  concentrates  on  scene  segmentation  based  on  general 
rules  about  objects  and  on  geometric  information  from  the  map  database.  Another  effort  related  to  image 
segmentation  is  that  by  Havens  and  Mackworth  [1983]  in  which  the  segmentation  process  is  guided  by  a 
description  of  map  information.  Sties,  et  al  [1977],  and  later  Kestner  and  co-workers  have  developed  an 
image-to-map  matching  concept  to  perform  similar  map-guided  image  segmentation  [Kestner,  1980;  Kestner 
and  Rumpler,  1984], 
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Medioni  and  Nevada  describe  a  recent  effort  to  automatically  match  two  aerial  images  with  each  other  and 
with  an  existing  map.  The  map  information  used  consists  of  object  boundaries.  The  map  database  itself  is 
not  the  focus  of  this  work,  nor  is  the  imaging  geometry  of  the  sensor  explicitly  modeled  [Medioni  and 
Nevada.  1984).  Both  of  these  aspects  of  the  problem  are  of  significance  for  our  proposed  efforts  during 
Phase  II. 

A  database  design  developed  in  Europe  has  been  used  to  explore  the  registration  of  aerial  photographs  [Leberl 
and  Ranzinger.  1982]  and  satellite  images  [Kropatsch  and  Leberl.  1981;  Ranzinger  and  Ranzinger,  1984], 
Other  work  concerned  with  the  exploitation  of  digital  map  databases  for  the  purpose  of  map-guided  feature 
extraction  or  photo-interpretation  must  also  necessarily  address  the  issue  of  map-guided  image  registration 
[Leberl  and  Kropatsch,  1979;  Ranzinger,  1985],  These  efforts  differ  from  the  work  of  other  authors  in  that 
the  database  is  searched  for  candidate  control  features,  the  sensor  geometry  is  modeled  and  the  search  for 
these  features  in  the  image  is  tailored  to  the  specific  features  of  interest. 

Image  registration  control  based  upon  non-point  features  is  of  particular  relevance  where  only  linear  and/or 
areal  features  exist.  Lugnani  [1982]  investigated  the  use  of  non-point  ground  control  in  conventional 
photograminetry;  the  control  information  in  this  work  consisted  of  linear  features.  Some  of  the  image 
matching  work  refered  to  earlier  [Medioni  and  Nevada.  1984]  also  explored  the  use  of  linear  features  for 
control,  as  does  the  work  on  image  rectification  done  by  Paderes,  et  al  [1984], 

The  use  of  representations  of  areal  features  for  matching  (as  opposed  to  simple  correlation  using  image 
segments  containing  those  features)  is  a  field  where  the  current  understanding  is  less  complete.  Landsat 
imagery  has  been  registered  to  a  map  in  work  done  by  Mokhtarian  and  Mackworth.  Their  method  represents 
areal  objects  with  a  ’generalized  scale  space'  image  which  is  invariant  under  rotation,  uniform  scaling  and 
translation  [Mokhtarian  and  Mackworth,  1986]. 

An  increasing  emphasis  on  the  use  of  rule-based  expert  systems  to  perform  automated  image  registration  and 
database  exploitation  is  apparent  [Robinson,  et  al,  1986],  The  development  of  rule-based  systems  to  perform 
photo-interpretation  is  the  subject  of  some  of  the  more  recent  work  with  the  MAPS  database  system 
[McKeown  and  McDermott,  1983;  McKeowm,  et  al,  1984].  A  more  general  system  called  LOBSTER  is  an 
attempt  to  integrate  a  Prolog-like  language  with  the  spatial  database  scheme  and  query  language  being 
developed  by  Frank  and  his  co-workers  [Frank.  1986]. 

5.3  Digital  Representation  of  Spatial  Data 

The  primary  concern  of  the  Phase  I  (and  subsequent)  effort  was  the  registration  of  imagery  not  to  other 
images,  but  to  a  digitally  stored  representation  of  the  region  or  surface  imaged.  A  number  of  methods  for 
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representing  spatial  data  have  been  developed.  These  include  a  several  variations  on  the  theme  of  spatial 
decomposition:  quadtrees,  hexagonal  structures,  etc..  Another  basic  technique  encodes  feature  information  in 
the  form  of  point  coordinates  and  vectors.  The  suitability  of  these  various  basic  data  formats  to  the  problem 
of  automatic  ground  control  generation  was  a  major  concern  of  the  Phase  1  work.  The  creation  and 
manipulation  of  such  representations  will  be  an  important  part  of  future  efforts. 

One  major  group  of  concepts  and  algorithms  for  the  storage  and  manipulation  of  spatial  data  centers  around 
the  idea  of  recursive  decomposition  of  the  area  or  volume  to  be  represented.  Reviews  of  quadtrees  and 
related  structures  have  been  presented  by  Samet  [Samet.  1984],  Although  quadtrees  in  their  basic  form  are 
strictly  areal  structures,  techniques  have  been  developed  to  allow  for  the  representation  of  linear  features  and 
the  boundaries  of  areal  features  with  them.  These  involve  a  number  of  conversion  algorithms  to  place 
boundary  (chain)  code  data  or  vector  data  into  a  quadtree  representation  (and  vice  versa)  [Samet  and  Webber, 
1984;  Dyer,  et  al,  1980;  Nelson  and  Samet.  1986],  The  efficiency  with  which  quadtree  data  can  be 
maintained  in  memory  may  be  increased  with  the  use  of  'linear  quadtrees’  [Gargantini,  1982],  A  quadtree¬ 
like  structure  for  representing  point  data  amounts  to  a  multiple-dimension  generalization  of  a  binary  search 
tree  [Finkel  and  Bentley,  1974],  A  further  refinement  of  this  idea  led  to  the  development  of  the  K-d  tree 
[Bentley.  1975J. 

The  decomposition  of  area  into  hexagonal  regions  rather  than  the  rectangular  regions  used  in  quadtrees  leads 
to  an  addressing  scheme  known  as  the  Generalized  Balanced  Ternary,  and  the  representation  of  areas  as 
hexagons  and  groups  of  hexagons  [Gibson,  ct  al.  1984].  The  decomposition  of  linear  features  into  ever 
smaller  segments,  each  one  represented  by  the  elongated  rectangle  that  contains  it,  leads  to  a  structure  known 
as  a  strip-tree  [Ballard,  1981b], 

Much  work  has  been  done  in  the  creation  and  use  of  databases  for  spatial  data.  Workers  at  Carnegie-Mellon 
University  have  developed  a  system  called  MAPS  (Map  Assisted  Photo-interpretation  System),  which  is  an 
attempt  to  integrate  image,  terrain  and  map  data  into  a  single  database  [McKeown,  1983;  McKeown,1984J. 
This  database  contains  image  segments  as  well  as  point  coordinates  of  landmarks  and  representations  of 
objects,  which  include  attributes  and  shape  descriptions  as  well  as  location. 

A  database  structure  that  is  based  on  a  regular  three-by-three  areal  decomposition  has  been  described  by 
Raetzsch  [1985].  This  ’nine-tree’  concept  (termed  an  areal  identification  tree)  is  similar  to  the  quadtree 
techniques  that  have  been  so  extensively  explored,  although  it  offers  some  computational  simplifications. 

An  extensive  set  of  software  has  been  developed  at  the  Jet  Propulsion  Laboratory  under  the  name  Image 
Based  Information  System  (IBIS).  It  allows  for  the  merging  of  image  data  and  data  in  the  form  of  maps 
(vectors)  and  tables.  Although  the  different  data  types  arc  maintained  separately,  the  non-image  data  is 


converted  into  a  raster  format  when  comparisons  or  combinations  are  needed.  This  supports  such 
applications  as  merging  thematically  classified  Landsat  data  with  georeferenced  data  to  report  land  cover 
distributions  {Angelici  and  Uryant,  I976J  and  the  digital  analysis  of  thematic  imagery  for  land  use  planning 
and  geological  study  [Farrell  and  Wherry.  1978;  Logan.  198 1 J.  Recently,  efforts  to  develop  a  database  query 
system  for  IBIS  that  will  allow  direct  Boolean  queries  have  been  made  [Friedman,  1982;  Friedman  1986). 

Database  systems  based  on  (joint  and  vector  coordinate  lists  have  been  examined  by  a  number  of  workers  in 
Europe  (e.g.  Kropatsch  and  Leberl  [1981]).  Point  coordinates  are  maintained  in  lists  ordered  so  that  the 
vectors  and  polygons  defined  by  those  points  are  represented  by  the  location  of  those  coordinates  in  the  list. 
Further,  pointers  are  used  to  link  related  vectors  represented  in  differing  places  in  the  coordinate  list. 

An  advanced  geographical  information  system  being  developed  at  Kork  Systems,  Inc  also  uses  a  point  and 
vector  format  with  associated  attributes  to  represent  spatial  features  [Keating,  et  al,  1986].  This  system 
makes  use  of  several  layers  of  data  organization  in  order  to  group  features  by  location  and  size.  An 
important  aspect  of  these  systems  is  their  use  of  internal  buffering  in  order  to  speed  memory  intensive 
operations.  A  query  language  for  the  efficient  retrieval  of  information  from  such  database  systems  is  an  area 
of  active  research  [Frank,  1982], 

5.4  Dead-Recknning  for  Image  Registration 

With  the  increasing  availability  of  satellite  ephemerides  to  users  of  remote  sensing  imagery,  the  published 
body  of  literature  on  the  registration  of  satellite  data  via  dead-reckoning  is  increasing.  This  is  deals  largely 
with  Landsat  and  Spot  imagery.  An  example  is  work  by  Puccinelli  [1976]. 

Radar  images  have  also  been  rectified  using  ephemeris  data.  Examples  are  by  Curlander  and  Brown  [1981], 
Curlander  [1984]  and  most  recently  by  Meier  and  Nuesch  [1986].  Essentially  this  work,  as  well  as  related 
efforts  with  other  types  of  digital  image  dead-reckoning,  concentrate  on  the  geometric  transformation  of  pixel 
coordinates  into  a  world  coordinate  system. 

Of  great  interest  is  the  source  of  navigation  data,  namely  GPS/INS  and  satellite  orbit  determination.  GPS  is 
the  most  widely  discussed  source  for  air  or  spaceborne  application.  An  excellent  review  paper  is  by  Hartl 
and  Wehr  [1986],  Recent  work  is  described  in  the  proceedings  of  symposia  of  ISPRS  Commission  I  [1986] 
and  on  Position,  Location,  and  Navigation  [1986],  including  GPS  reports  by  Klensberg  and  Wells  [1986],  A 
broader  review  of  navigation  data  has  been  authored  by  Corten  [1984]. 

GPS  applications  to  image  dead-reckoning  have  not  come  to  our  attention.  INS  uses  have  been  reported  early 
on  (e.g.  Roessel  and  de  Godoy,  [1974])  but  concentrated  more  on  navigation  than  on  rectification.  VEXCEL 


is  currently  under  contract  to  INTERA  Technologies  Ltd.  to  develop  a  capability  to  rectify  aircraft  SAR 
images  using  on  board  INS  observation.  This  work  is  in  progress.  The  ultimate  intent  is  to  also  include 
GPS  data. 


5.5  SAR  Imagery  Registration 

i  We  have  referred  in  several  instances  to  SAR-map  and  SAR-SAR  registration  in  various  parts  of  this 

proposal.  The  question  has  been  addressed  by  numerous  authors  in  the  field  as  documented  in  SAR  review 
papers  [Leberl.  I983J  and  recent  research  articles  [McConnell.  I987J.  The  major  difficulty  with  SAR  in  a 
registration  task  is  the  geometric  resolution  and  dependence  of  image  contents  on  platform  position.  A  recent 
major  effort  dealt  with  Shuttle  SIR-B  digital  SAR  data.  Multiple  images  were  co-registered  to  create  new  data 
sets  (Domik,  et  al,  1986).  SIR-B  has  become  a  major  source  of  experimental  SAR  data. 

The  dissimilarity  of  SAR  images  as  a  result  of  sensor  positions  has  been  addressed  in  work  reported  by 
Fullerton,  et  al  [1986].  Attempts  were  made  to  remove  the  dissimilarities.  This  was  successful  for  large 
^  features,  but  could  not  achieve  success  with  image  micro-detail. 

1 

5.6  Infrared  Imagery 

Geometric  work  on  infrared  imagery  is  scarce  in  the  open  literature.  Infrared  line  scanning  was  modeled  by 
Konecny  [1971],  Leberl  [1972]  and  others.  However,  application  scenarios  that  would  drive  a  geometric 
rectification  or  registration  effort  have  yet  to  develop. 

Conceptually,  infrared  imagery  is  obtained  in  one  of  two  ways.  The  first  is  by  optical-mechanical  scanning: 
the  method  is  thus  geometrically  identical  to  Landsat-MSS  type  sensing.  Otherwise,  a  framed  image  is 
created  where  the  focal  plane  of  an  optical  system  is  scanned.  In  that  case  the  geometric  model  is  a  central 
perspective. 

The  uniqueness  of  infrared  imagery  is  therefore  not  in  the  geometry  but  in  the  radiometric  properties. 
Figure  5.1  illustrates  this  by  presenting  night  time  and  day  time  infrared  images  and  compares  this  to  a 
panchromatic  aerial  photograph  (all  images  being  of  the  same  scene).  Figure  5.2  is  another  example 
showing  a  day  time  infrared  image  of  a  lake  with  traces  of  a  boat  that  distributed  the  lake  surface. 

Infrared  imagery,  through  efforts  to  automate  the  recognition  of  military  targets,  has  been  subject  to  extensive 
pattern  recognition  research.  This  is  documented  in  extensive  technical  reports,  for  example  by  Milgram.  et 
al  [1978).  However,  the  work  does  not  address  registration  or  rectification. 
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Figure  5.1:  Night  time  (a)  and  daytime  (b)  infrared  image  of  a  lake,  and  comparison  with  a  panchromatic 
photograph  (c).  Note  the  temperature  reversal  between  lake  and  land  from  day  to  night. 
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Figure  5.2:  Example  of  an  airborne  infrared  image  of  a  lake  showing  traces  of  a  boat.  Note  the  brightness 
of  the  south-facing  shape  due  to  sun-heat. 
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5.7  Conclusion 


The  implementation  of  a  database  system  to  generate  ground  control  data  for  image  registration  is  a  natural 
outgrowth  of  the  recent  research  in  spatial  database  design,  image  matching  and  expert  system  development. 
Clearly,  the  primacy  of  the  registration  task  makes  this  an  obvious  realm  for  initial  attempts  to  create 
working,  productive  systems  (as  opposed  to  research  testbeds).  Additionally,  the  exploitation  of  existing 
spatial  database  design  concepts  through  the  use  of  rule-based  systems  is  a  Held  of  endeavor  that  is  ripe  for 
exploration  and  development.  The  relatively  straight-forward  nature  of  the  knowledge  involved  in  choosing 
candidate  ground  control  features  from  a  spatial  database  makes  this  particular  function  an  appealing  place  to 
begin  this  development. 
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Prototype  Specifications 


While  a  detailed  analysis  of  the  process  by  which  a  prototype  of  the  Smart  MC&G  Control  Generator  will  be 
created  during  Phase  II,  the  basics  of  a  viable  approach  seem  clear.  The  emphasis  in  this  approach  is  on 
developing  the  general  spatial  data-handling  capabilities  needed  in  such  a  system  without  restricting  it  to  a 
specific  set  of  current  sensor  attributes,  image  matching  techniques,  or  applications.  Thus,  this  prototype 
could  serve  not  only  to  develop  the  capability  of  using  current  technology  to  register  imagery,  but  also  to 
explore  the  utility  of  new  sensor  and  image  matching  methods. 

The  heart  of  the  system,  the  spatial  database  itself,  should  be  built  from  an  existing,  highly-structured 
point/vector  format  database  system.  Further,  this  system  should  allow  for  the  maintenance  and  manipulation 
of  features  as  objects,  comprising  the  geometric  and  attribute  data  of  interest  as  single,  coherent  units.  It 
should  employ  feature  clustering  techniques  to  minimize  memory  space  and  disk  access.  This  clustering 
should  be  according  to  feature  size  as  well  as  location  and  type.  At  least  one  such  system  is  currently  being 
developed  in  a  form  that  appears  to  suit  the  basic  requirements;  it  is  likely  that  others  could  also  be  found. 

The  most  important  aspect  of  the  prototype  development  will  be  the  construction  of  the  rule-based  query 
generator.  This  will  be  both  the  mechanism  by  which  the  database  is  exploited,  and  the  primary  source  of 
interaction  between  the  control  generator  and  a  human  operator  or  other  software  modules.  This  portion  of 
the  control  generator  will  also  calculate  the  propagation  of  error  estimates  to  be  reported  along  with  the 
candidate  ground  control.  Because  these  estimates  will  be  dependent  on  knowledge  of  the  sensor  and 
matcLitg  techniques  encoded  in  the  rule  system,  the  query  generator  is  the  appropriate  module  in  which  to 
house  this  functionality. 

The  digital  elevation  modeling  capability  needed  for  the  control  generator  will  be  incorporated  into  the  spatial 
database  system  described  above.  This  work  need  not  be  done  prior  to  the  development  of  the  rule-based 
query  generator.  Proper  implementation  of  the  knowledge  base  will  allow  for  efficient  updating  of  the  rule 
system,  so  that  the  additional  capabilities  supported  by  the  DEM  functionality  can  be  incorporated  after  the 
initial  rule  and  meta-rule  development.  However,  the  most  efficient  development  of  the  Smart  MC&G 
Control  Generator  would  involve  the  integration  of  the  DEM  capability  into  the  database  system  prior  to  (or  at 
least,  concurrent  with)  the  construction  of  the  knowledge  base  and  query  generator  interfaces. 
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These  basic  steps  of  acquisition  and  modification  of  an  existing  advanced  spatial  database,  integration  of 
digital  elevation  modeling  functionality,  and  construction  of  rule-based  query  generator  and  interface  modules 
would  result  in  the  rapid  development  of  a  prototype  control  generator.  Further,  this  approach  retains  a 
generality  and  flexibility  of  function  that  will  make  the  system  useful  beyond  the  currently  envisioned  purpose 
of  generating  control  data  from  existing  sensor  and  image  technology. 
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Conclusion 

The  basic  purpose  of  this  work  was  to  evaluate  the  feasibility  and  utility  of  constructing  a  Smart  MC&G 
Control  Generator,  and  to  define  the  basic  characteristics  it  should  exhibit.  The  utility  of  such  a  system  is 
self-apparent:  a  working  control  generator  would  greatly  increase  the  speed  with  with  which  imagery  of 
differing  kinds  could  be  registered  to  a  common  coordinate  system.  This  would  allow  not  only  the  updating 
of  an  existing  map  data  base,  but  the  exploitation  of  this  imagery  for  a  multitude  of  operations,  including 
feature  detection/extraction,  target  acquisition  and  tracking,  etc..  Additionally,  this  control  generator  would 
enhance  the  comparison  and  evaluation  of  new  techniques  of  image  matching  and  the  integration  of  new 
sensor  technology  into  existing  applications. 

The  development  of  such  a  system  seems  more  than  simply  feasible,  in  many  respects,  it  appears  to  be  the 
logical  next  step  in  both  automated  image  registration  and  in  the  exploitation  of  spatial  databases  using  some 
of  the  recently  developed  tools  of  artificial  intelligence.  The  primary  nature  of  the  task  of  automated  ground 
control  generation,  with  its  inherent  dependence. on  both  the  imagery  collected  and  the  digital  representation 
of  the  information  in  that  imagery,  places  it  central  to  many  applications  of  remote  sensing.  Further, 
recently  developed  spatial  databases  appear  to  have  many  of  the  attributes  necessary  for  the  task. 

The  restricted  and  relatively  well-defined  character  of  the  knowledge  base  involved  in  choosing  candidate 
ground  control  makes  this  function  suited  to  the  use  of  rule-based  systems  and  to  the  currently  problematical 
task  of  developing  such  systems.  The  use  of  expert  system  techniques  in  constructing  the  prototype  system 
will  have  the  additional  benefit  of  affording  background  and  experience  in  the  use  of  knowledge-based  systems 
in  image  and  database  exploitation,  an  area  which  will  surely  be  important  to  the  future  development  of 
remote  sensing  applications. 

In  conclusion,  we  find  that  the  development  of  a  Smart  Mapping.  Charting  and  Geodesy  Control  Generator  is 
a  job  that  both  could  and  should  be  done. 


Appendix 


DTED/DFAD  Dal;i  for  Automated  Ground  Control 


An  experiment  was  performed  to  gain  a  basic  understanding  of  the  utility  of  existing  Defense  Mapping 
Agency  data  in  generating  ground  control  for  different  image  types.  Specifically.  Digital  Terrain  Elevation 
Data  (DTED)  and  Digital  Feature  Analysis  Data  (DFAD)  was  examined  with  respect  to  its  usefulness  in 
registering  high  altitude  photography  and  satellite-borne  synthetic  aperture  radar  (SaR)  images. 

For  the  area  examined,  we  find  that  the  terrain  model  represented  by  DTED  data  is  quite  poor,  and  of  limited 
utility  for  control  generation.  This  is  in  comparison  to  the  terrain  model  produced  by  the  analysis  of 
photographic  stereo-pairs.  The  density  of  data  points  and  the  accuracy  of  interpolation  routines  used  in 
developing  the  DTED  data  set  for  a  given  area  is  apparently  inadequate  for  modeling  the  high  spatial 
frequency  features  (e.g.  small  ridges,  valleys,  hilltops,  etc.)  needed  for  image  registration  control.  The 
DFAD  data  is  even  more  sparse,  containing  few  if  any  features  that  are  both  suitable  for  control  and  visible 
in  the  images. 


The  Data  Set 

The  basis  for  choosing  the  particular  data  set  used  in  this  experiment  was  the  ready  availability  of  the 
differing  data  types  used.  A  region  of  approximately  12  kilometers  by  13  kilometers  in  north-central  Georgia 
was  chosen.  A  swath  of  Georgia  covered  by  the  second  Shuttle  Imaging  Radar  experiment  (SIR-B.  orbited 
by  the  shuttle  Challenger  in  October  1984)  was  examined  in  order  to  identify  a  region  containing  buildings 
roads  and  other  features  that  could  be  expected  to  be  represented  in  the  DFAD  data.  Upon  identification  of  a 
suitable  region  (33°53'05"  through  34°00'42"  north  latitude.  83°37'58”  through  83°46'04"  west 
longitude),  the  following  data  was  collected: 

*  DTED  data  covering  the  region 

*  DFAD  data  covering  the  region 

*  National  High  Altitude  Photographs  (NIIAP)  of  the  region 

*  SIR-B  data  covering  the  region 

*  USGS  topographic  maps  of  the  region 


The  DTED  data  consists  of  a  grid  of  elevation  values,  separated  from  one  another  by  three  arc  seconds  in 
both  latitude  and  longitude.  At  this  latitude,  this  amounts  to  a  nearly  square  grid  of  points  92.5  meters  apart. 
The  features  represented  in  the  L)I:AD  data  set  covering  this  region  include  several  lakes,  a  dam,  an  airport, 
an  urban  area,  and  a  number  of  isolated  buildings.  The  NHAP  photographs  are  1:80.000  scale  photos  with 
overlapping  coverage  so  that  stereo  analysis  is  possible.  The  SIR-B  data  is  SAR  imagery  with  a  pixel 
resolution  of  12.5  meters.  Finally,  the  topographic  maps  used  were  1:24.000  scale,  updated  from  aerial 
photographs  taken  in  1981. 


Processing 

The  first  step  in  examining  the  DTED  terrain  model  was  to  create  a  second  model  with  which  it  could  be 
compared.  This  was  done  by  collecting  a  grid  of  elevation  points  from  stereo-pairs  of  the  NHAP  photos  with 
an  analytic  stereo  plotter.  Each  set  of  elevation  data  was  used  to  interpolate  a  grid  of  three-dimensional 
coordinate  data  from  which  the  terrain  surface  could  be  modeled.  The  spacing  of  this  grid  was  the  same  in 
each  case  (96  meters  square),  and  the  interpolation  routine  used  was  the  same.  Thus,  any  differences  in  the 
models  derived  from  the  two  data  sets  can  be  ascribed  to  the  input  data  itself,  and  not  to  subsequent 
differences  in  processing. 

A  contour  map  was  derived  from  this  grid  in  order  to  check  the  quality  of  the  data  collection  and  elevation 
interpolation  routines  (Figure  A-l).  This  contour  map  shows  very  good  correlation  with  the  USGS 
topographic  maps,  indicating  that  the  terrain  model  derived  from  the  photographs  is  quite  accurate.  Portions 
of  the  USGS  map  and  of  the  contour  map  produced  from  the  photograph-derived  terrain  model  are  shown  in 
Figures  A-2  and  A-3.  These  portions  arc  at  the  same  scale,  so  a  direct  comparison  can  be  made.  The 
coordinate  system  used  in  all  of  these  models  is  the  Universal  Transverse  Mercator  (UTM)  system.  The 
photo-derived  model  was  initially  created  with  this  coordinate  system,  while  the  DTED-derived  model  was 
converted  to  it  from  its  initial  latitude-longitude  system. 

Perspective  views  of  the  photo-derived  elevation  model,  the  DTED-derived  model,  and  a  difference  model 
were  then  constructed.  These  perspective  views  can  be  seen  in  Figures  A-4,  A-5  and  A-6.  The  terrain 
models  constructed  cover  a  region  13.7  kilometers  by  12.8  kilometers  in  area.  The  perspective  views 
presented  here  show  these  models  as  seen  from  above  and  to  the  southeast  of  the  region.  Due  to  the  low 
relief  in  the  region  (105  meters  elevation  difference  between  the  lowest  and  highest  points),  the  vertical  scale 
in  these  views  is  substantially  exaggerated. 

Figures  A-7  and  A-8  show  orthographic  views  of  the  photo-derived  and  DTED-derived  DEM.  respectively. 
These  views  are  oriented  such  that  north  is  toward  the  top  of  the  image,  and  the  illumination  source  from  the 
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northwest.  at  45°  from  the  zenith.  The  vertical  scale  of  these  images  is  exaggerated  by  the  same  factor  as  in 
the  perspective  views. 

Next,  both  the  photo  image  and  the  radar  image  were  registered  to  the  coordinate  system  of  the  DEMs.  This 
involved  finding  points  in  each  image  that  were  identifiable  on  the  USGS  topographic  map.  This  task  was 
easily  accomplished  in  the  case  of  the  aerial  photograph.  Prospective  points  were  chosen  from  the  original 
film  image,  and  then  the  digitized  image  was  examined  to  discover  the  pixel  coordinates  of  these  points 
corresponding  to  the  known  map  (UTM)  coordinates.  Additionally,  those  portions  of  the  original  photo  that 
extend  beyond  the  edges  of  the  DEM  were  removed.  An  affine  transformation  was  then  calculated  to  warp 
the  digital  photo  image  of  the  DEM  region  to  the  DEM  coordinate  system.  Figure  A-9  shows  the  original, 
unregistered,  digitized  photo  image,  while  Figure  A-l  I  is  of  the  cropped  and  registered  photo  image. 

A  similar  process  was  used  to  register  the  digital  radar  image.  However,  the  identification  of  control  points 
in  the  radar  image  was  a  problem  of  considerable  difficulty.  Indeed,  only  a  single  point  could  identified  in 
both  the  radar  image  and  the  map.  All  other  ground  control  was  found  by  areal  correlation  with  the  human 
eye.  This  was  done  by  constructing  hard  copy  (on  film)  images  of  the  radar  images  at  the  same  scale  as  the 
map.  These  images  can  then  be  placed  over  the  map  and  manipulated  until  an  experienced  observer 
determines  that  obscure  features  and  patterns  in  the  radar  image  are  positioned  directly  over  corresponding 
features  in  the  map.  This  areal  correlation  is  transformed  into  control  point  information  by  simply  marking 
the  map  (pinpricks  through  image  and  map  were  actually  used)  in  locations  where  individual  pixels  in  the 
radar  image  are  identifiable.  Figure  A-IO  is  of  the  original,  unregistered  radar  image,  and  Figure  A-12 
shows  the  portion  of  the  radar  image  (registered)  covered  by  the  DEMs. 

Figures  A- 1 3  and  A- 14  indicate  the  warping  which  was  done  to  the  photo  image  in  order  to  rectify  it  to  the 
DEMs.  Figure  A-13  is  simply  a  regular  grid,  while  A-14  shows  the  result  of  applying  to  that  grid  the  same 
affine  transformation  used  to  rectify  the  photo.  Notice  that  the  transformation  was  basically  composed  of  a 
scale  change  and  slight  rotation.  Slight  irregularities  in  the  grid  structure  of  Figure  A-14  indicate  local 
distortions  (with  respect  to  the  DEMs)  in  some  regions  of  the  photograph. 

Once  the  photo  and  radar  images  are  registered,  various  combined  views  of  the  images  and  DEMs  are 
possible.  In  Figure  A- 1 5.  the  rectified  photo  image  is  overlain  atop  the  photo-derived  DEM  -  in  perspective 
view.  Similarly,  the  rectified  radar  image  is  overlain  on  that  same  perspective  view  of  the  DEM  in  Figure  A- 
16.  In  Figures  A-17  through  A-20,  the  elevation  information  is  represented  by  the  color  in  the  image  (an 
intensity/hue/saturation  transformation).  Low  elevations  are  blue  and  increasing  elevations  progress  through 
the  color  spectrum  to  red.  This  display  technique  preserves  more  of  the  information  in  the  image  itself  than 
docs  the  perspective  view  overlay.  In  Figure  A-17.  the  photo  image  is  transformed  with  respect  to  the  photo- 
derived  DEM,  while  in  Figure  A- 1 8  the  DTED-dcrived  DEM  is  used.  Similarly.  Figures  A- 19  and  A-20 
show  the  radar  image  combined  with  the  two  elevation  models. 


Discussion 


The  difference  between  the  photo-derived  and  DTED-derived  elevation  models  is  quite  striking.  The  DTED 
model  (Figures  A-5  and  A-8)  represents  only  the  broad,  general  aspects  of  the  region's  topography.  The 
valley  in  which  the  southeast-trending  Marbury  Creek  is  apparent,  as  is  the  eastward  trending  Apalachee 
River  valley  in  the  southern  portion  of  the  region.  However,  the  smaller  gullies  and  ridges  formed  by 
numerous  tributary  creeks  are  largely  missing  in  the  DTED  model.  In  contrast,  the  photo-derived  model 
(Figures  A-4  and  A-7)  preserves  these  smaller  features  quite  well.  These  numerous  small  features  obscure 
(to  the  human  eye)  the  general  topographic  character  of  the  region  when  compared  to  the  DTED  DEM,  but 
close  inspection  of  the  image  shows  that  this  information  is  retained. 

The  appearance  of  the  perspective  view  of  the  difference  DEM  (Figure  A-6).  also  indicates  this  lack  of  high 
frequency  information  in  the  DTED  model.  The  difference  DEM  appears  quite  like  the  photo-derived  DEM. 
indicating  that  the  DTED  model  contains  relatively  few  elevation  changes,  and  large  areas  of  nearly  constant 
elevation.  It  is  these  smaller  features  that  would  be  most  useful  in  generating  features  (radar  shadows, 
intensity  contrasts,  etc.)  in  simulated  radar  images  that  would  be  likely  to  be  visible  in  actual  radar  data. 
Thus,  it  seems  that  the  DTED-generated  DEM  would  be  of  minimal  utility  for  registering  radar  imagery 
through  the  use  of  areal  or  edge  correlation  with  simulated  images. 

The  elevations  present  in  the  elevation  model  derived  front  aerial  photography  range  from  a  low  of  207 
meters  to  a  high  of  312  (a  range  of  105  meters).  The  DTED-derived  model  exhibits  elevation  values  from 
214  meters  to  305  meters  (a  range  of  91  meters),  indicating  that  the  highest  and  lowest  features  of  the  photo- 
derived  model  arc  not  represented  in  the  DTED  model.  The  values  of  the  difference  DEM  (DTED  model 
subtracted  front  the  photo-derived  model)  range  from  -34  meters  to  48  meters.  Notice  that  the  average 
elevation  of  both  the  DTED  and  photo  models  is  the  same  (259.5  meters),  while  the  average  of  the  difference 
model  is  7.0  meters.  This  is  probably  due  to  the  difficulty  the  DTED  model  has  in  representing  small 
regions  of  high  elevation  (peaks  and  ridges),  as  discussd  above. 

Close  inspection  of  the  orthographic  views  of  the  two  DEMs  (Figures  A-7  and  A-8)  reveals  additional 
problems  with  the  DTED  model.  Distinct  steps  can  be  seen  in  regions  of  changing  elevation  (i.e.  slopes). 
This  artifact  of  the  data  results  in  representing  sloping  areas  with  a  staircase-like  shape,  rather  than  the  more 
continuous  elevation  change  seen  in  the  photo-derived  model.  Again,  this  feature  of  the  DTED  model  would 
present  problems  to  a  radar  image  simulation  technique  that  accounted  for  the  angle  of  incidence  between  the 
radar  energy  and  the  reflecting  surface. 

The  difficulty  experienced  in  identifying  ground  control  in  the  radar  image  served  to  emphasize  the 
problematic  nature  of  automating  such  a  task.  The  specific  image  we  worked  with  was  not  an  example  of  the 
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best  quality  SIR-B  data:  however,  it  was  not  atypicalh  poor  either.  The  inability  of  experienced  radar  image 
specialists  to  identify  specific  points  or  even  linear  features  in  this  image  indicates  that  such  information  will 
not  always  be  useful  for  automated  registration.  Our  eventual  reliance  on  areal  correlation  via  the  human  eye 
suggests  that  approaches  involving  the  correlation  of  image  segments  with  synthetic  segments  generated  from 
ground  (map)  information  should  be  thoroughly  explored  for  their  utility  to  automated  control  generation. 
Such  approaches  would  require  that  feature  and  attribute  data  relevant  to  radar  backscattter  levels  be 
maintained  in  a  Smart  Control  Generator's  geographical  database. 

The  discovery  of  ground  control  points  proved  a  much  simpler  task  in  registering  the  NHAP  photograph  to 
the  DEMs.  Numerous  features  were  found  on  the  map  (and  thus,  would  exist  in  a  Smart  Control 
Generator's  database)  that  were  expressed  as  distinct  intensity  differences  (i.e.  edges  or  points)  in  the 
digitized  photo.  Areal  correlation  was  unnecessary,  suggesting  that  a  process  of  image  simulation  from  map 
data  would  be  unimportant  for  most  if  not  all  photographic  image  registration  tasks. 

Inspection  of  the  DFAD  data  from  this  region  revealed  few  features  that  were  identifiable  in  the  photograph. 
An  urban  area  (Winder.  GA)  is  outlined  in  the  DFAD  database,  but  those  outlines  bear  little  resemblance  to 
city  edges  visible  in  the  image.  The  large  lake  (Marbury  Lake?)  is  present  in  the  DFAD  data.  Correlation 
of  this  shape  with  that  region  of  the  image  would  produce  usable  control  information;  either  a  single  point 
related  to  the  area  itself  (e.g.  centroid),  or  several  points  found  as  inflections  in  the  lake's  boundary.  An 
airport  is  represented  in  the  DFAD  data  as  single  lines  running  parallel  to  the  runway  axes.  This 
information  is  inadequate  for  choosing  single  pixel  points  in  the  photo  image,  unless  other  information  about 
the  relationship  between  the  lines  in  the  DFAD  and  the  actual  runway  edges  is  known.  The  other  features 
found  in  the  DFAD  data  consist  of  isolated  buildings.  These  are  difficult  to  identify  in  the  image  without  a 
preliminary  knowledge  of  their  approximate  location.  Thus,  such  features  may  be  useful  for  refinement  of  an 
image  registration  given  a  preliminary  image  to  map  correspondence.  The  map  features  most  useful  for 
control  generation  -  road  and  highway  intersections  and  inflections,  power  line,  pipeline  and  creek 
intersections,  etc.  -  were  not  represented  in  the  DFAD  data. 

The  use  of  these  DFAD  features  for  registration  of  the  radar  image  is  even  less  promising  than  in  the  case  of 
the  NHAP  photo.  Even  the  large  lake,  the  only  really  useful  feature  for  photo  registration,  appears  quite 
different  in  shape  in  the  radar  image.  This  results  from  a  lowered  water  level  in  the  reservoir,  caused  by 
increased  water  demand  in  the  region  due  to  drought  during  the  period  of  SIR-B  data  collection.  Thus,  we 
found  no  features  in  the  DFAD  data  that  could  be  used  to  aid  in  registration  of  this  SIR-B  image. 
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Conclusion 


In  conclusion,  we  find  th.it  the  Digital  Terrain  Elevation  Data  only  generally  describes  the  topography  of  the 
region  examined.  The  details  important  to  the  task  of  radar  image  simulation  and  registration  are  largely 
lacking.  Further,  even  this  general  (low  frequency)  model  possesses  artifacts  that  would  coomplicate  the 
image  simulation  process.  It  appears  that  DTED  data  wilt  be  useful  only  for  crude,  preliminary  registration 
tasks. 

The  utility  of  Digital  Feature  Analysis  Data  for  automated  registration  also  appears  severely  limited.  The 
absence  from  the  DFAD  database  of  the  cultural  features  most  useful  for  control  information  is  its  greatest 
drawback.  However,  the  possibility  exists  that  this  data  may  be  useful  for  certain  registration  tasks.  Some 
permanent  areal  features  may  be  used  to  generate  control  points.  (Note  that  we  were  fortunate  in  this 
experiment  that  the  NHAP  photograph  and  the  DFAD  data  were  prepared  at  times  when  the  lake  possessed 
similar  water  levels  and  boundaries.)  Single  buildings  may  be  useful  for  registration  refinement. 

In  general  it  seems  that  a  Smart  MC&G  Control  Generator  will  require  geographical  information  that  is  more 
current,  more  detailed,  and  more  specifically  oriented  toward  control  generation  than  the  DTED  and  DFAD 
data  examined  here. 


Figure  A- 1  Contour  Map  of  the  Digital  Elevation  Model  (DEM)  derived  from  NHAP  photographs  of  the 
region  of  this  study.  This  map  is  drawn  at  a  scale  of  l:b().(X)0  and  with  a  contour  interval  of  20  feet  (6.1 
meters).  All  data  were  treated  simply  as  terrain  points;  that  is.  special  point  tvpes  (such  as  those  on 
breaklines,  formlines  or  those  delineating  lakes)  were  not  treated  dilferentlv  than  ordinary  latitude-longitude- 
elevation  measurements.  The  corner  coordinates  seen  are  in  the  Universal  Transverse  Mercator  Svstem 


Figure  A-4  A  perspective  view  of  the  photo-derived  DEM.  This  view  is  from  above  and  to  the  southeast  of 
the  region.  The  following  views  of  the  DTED-derived  and  difference  DEMs  (Figures  A-5  and  A-6)  are  from 
the  identical  viewpoint.  Note  the  relatively  detailed  nature  of  this  view.  Many  small  gullies  and  ridges 
formed  by  tributaries  to  the  major  streams  can  be  clearly  seen.  In  this  and  all  subsequent  views  of  the 
DEMs.  the  elevations  have  been  exaggerated.  The  terrain  in  this  area  is  actually  quite  flat,  with  only  105 
meters  elevation  difference  between  the  lowest  and  highest  points. 
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Figure  A -5  A  perspective  view  of  the  DTED-derived  DEM.  This  figure  shows  the  general  nature  of  the 
topographic  information  contained  in  this  DTED  data.  The  major  river  valleys  can  be  discerned,  but  the 
smaller  tributary  features  are  lost.  Large  areas  of  fine  detail  in  the  photo-derived  DEM  appear  only  as  flat, 
featureless  plateaus  or  valley  floors  in  the  DTED-derived  model. 
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Figure  A-8  The  orthogonal  view  of  the  DTED-derived  DEM.  Comparison  of  Figures  A-7  and  A-8 
illustrates  some  additional  deficiencies  in  the  DTED  data.  Note  the  stair-step  appearance  of  the  slopes  in  the 
DTED  model  (particularly  visible  in  the  southeast  corner).  This  indicates  a  lack  of  resolution  in  the  elevation 
data  that  would  make  simulation  of  the  radar  return  (for  the  purpose  of  image  registration)  from  these  slopes 
difficult. 


Figure  A-ll  The  portion  of  the  aerial  photograph  (NHAP)  covered  by  the  elevation  models.  This  image  has 
been  registered  to  the  DEMs  using  an  affine  transformation.  The  transformation  required  to  register  this 
image  consisted  basically  (globally)  of  a  scale  change  and  a  slight  rotation  (see  Figures  A-13  and  A-14). 


Figures  A-13  and  A-14  These  figures  illustrate  the  transformation  required  to  register  the  NHAP  photograph 
to  the  elevation  models.  The  red  grid  has  been  subjected  to  that  same  transformation  to  produce  the  green 
grid.  Notice  that  the  overall  change  involves  a  slight  clockwise  rotation  and  a  substantial  scale  increase. 
Close  inspection  reveals  smaller  local  distortions  in  the  grid,  representing  local  warping  of  the  image  to  the 
elevation  model. 
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Figure  A- 1 7  The  registered  NHAP  photograph.  In  this  image  and  in  the  ones  that  follow  (Figures  A-18,  A- 
19,  A-20),  the  elevation  model  is  represented  by  the  color  in  the  figure.  The  highest  elevations  are  in 
magenta,  and  the  lowest  in  dark  blue,  while  the  intermediate  elevations  are  represented  by  the  color  spectrum 
between  the  two  extremes.  This  figure  shows  the  photo-derived  elevation  model  in  this  way,  while  the  next  is 
based  on  the  DTED-derived  DEM. 


Figure  A-18  The  registered  NHAP  photo  colored  to  reflect  the  DTED-derived  DEM.  Note  the  relatively 
slight  variation  in  colors,  indicating  a  lack  of  small  (high  frequency)  terrain  features. 


62 


References 


Angelici,  G.L.  and  Bryant,  N.A.,  1976.  "Techniques  for  the  Creation  of  Land  Use  Maps  and  Tabulation 
form  Landsat  Imagery",  Proc.  Second  Annual  William  T,  Pecora  Mem  Svmp,.  Sioux  Falls.  SD 

Anuta,  P.E,  1970,  "Spatial  Registration  of  Multispectral  and  Multitemporal  Digital  Imagery  Using  Fast 
Fourier  Transform  Techniques",  IEEE  Trans.  Geosci.  Elect.,  vol.  8,  no.  4 

Ballard,  D.H.,  1981a.  "Generalizing  the  Hough  Transform  to  Detect  Arbitrary  Shapes".  Pattern 
Recognition,  vol.  13,  no.  2 

Ballard,  D.H.,  1981b,  "Strip  Trees:  A  Hierarchical  Representation  for  Curves",  Comm,  of  the  ACM,  vol. 
24,  no.  5 

Ballard,  D.H.  and  Brown,  C.M.,  1982.  Computer  Vision.  Prentice-Hall 

Barnea,  D.I  and  Silverman,  H.F.,  1972.  ”A  Class  of  Algorithms  for  Fast  Digital  Image  Registration",  IEEE 
Trans.  Computers,  vol.  21,  no..  2 

Barrow,  H.G.,  et  al,  1977,  "Experiments  in  Map-Guided  Photo  Interpretation",  Proc.  Fifth  Inti.  Conf.  on 
Art.  Int.,  Cambridge,  MA 

Bentley,  J.L.,  1975,  "Multidimensional  Binary  Search  Trees  Used  for  Associative  Searching",  Comm,  of  the 
ACM,  vol.  18,  no.  9 

Burt,  P.J.,  1982,  "Pyramid-Based  Extraction  of  Local  Image  Features  with  Applications  to  Motion  and 
Texture  Analysis",  SPIE  Conf,  on  Robotics  and  Industrial  Inspection.  San  Diego,  CA 

Burl,  P.J.,  1983,  "The  Pyramid  as  a  Structure  for  Efficient  Computation",  in  Multi-Resolution  Inage 
Processing  and  Analysis.  (Rosenfeld,  ed.),  Springer-Vet  lag 

Corten,  F.L.I.H.,  1974,  "Applications  of  Navigation  Systems  and  of  Sensor  Orientation  Systems  in  Survey 
Navigation,  in  Aerial  Triangulation  and  in  Establishment  of  Control".  Proc.  of  ISPRS  Congress.  Bio.  1984. 
Comm.  I,  Working  Group  3 


Frank,  A.U.,  1986.  LOBSTER:  Combining  Database  Management  and  Artificial  Intelligence  Techniques  to 
Manage  Land  Information.  Report  No.  56.  Surveying  Eng.,  Dept,  of  Civil  Eng..  Univ.  of  Maine.  Orano. 
ME 


Frledinnit,  S.Z.,  1982.  Image  Based  Approach  to  Mapping,  Charting  and  Geodesy.  Publication  715-158  (D- 
1689).  Jet  Propulsion  Laboratory 

Friedman,  S.Z.,  1986.  IBIS  Query:  Software  to  Support  the  IBIS  Expansion  for  Mapping.  Charting  and 
Geodesy.  Report  No.  ETL-0422.  U  S.  Army  Engineer  Topographic  Laboratories 

Fullerton,  K.,  et  al,  1986.  "Opposite  Side  SAR  Image  Processing  for  Stereo  Viewing",  Photogrammetric 
Eng,  and  Remote  Sensing,  vol.  LII.  no.  9 

Gargantini,  I.,  1982.  "An  Effective  Way  to  Represent  Quadtrees",  Comm,  of  the  ACM,  vol.  25,  no.  12 

Gibson,  L.,  ct  al,  1984,  Hexagonal  Data  Base  Study  (Phase  II).  Report  No.  ETL-360,  U.S.  Army  Engineer 
Topographic  Laboratories 

Grimson,  W.E.L.,  1985,  "Computational  Experiments  with  a  Feature  Based  Stereo  Algortlim",  IEEE-PAMI, 
vol.  7,  no.  1 


Guindon,  B.  and  Maruyaina,  H.,  1986,  "Automated  Matching  of  Real  and  Simulated  SAR  Imagery  as  a 
Tool  for  Ground  Control  Acquisition",  Canadian  Journal  of  Remote  Sensing,  vol.  12.  no.  2 

Hard,  P.H.  and  Welir,  A.,  1986,  "Chanccn  der  GPS-Satellitennavigation  fuer  die  Luft  Photogram  metrie", 
Bildmcssung  und  Luftbildwesen,  6/86 

Havens,  W.L.  and  Mnckworth,  A.,  1983,  "Representing  Knowledge  of  the  Visual  World".  lEEE-Computer. 
Oct.  1983 

Hough,  P.V.C.,  1962,  "Method  and  Means  for  Recognizing  Complex  Patterns",  U.S.  Patent  No.  3,069,654 

Hueckel,  M.,  1971,  "An  Operator  Which  Locates  Edges  in  Digitized  Pictures",  Journal  of  the  ACM,  vol. 
18,  no.  I 

Keating,  T.,  el  al,  1986,  Database  Design  Specific  to  Geographic  Information  Systems.  Kork  Systems,  Inc.. 
Bangor,  ME 

Kestner,  W.,  1980,  "Considerations  About  Knowledge-Based  Image  Interpretation",  Proc.  IAPR.  Miami,  FL 

66 


Kestncr,  W.  and  Ruiuplcr,  C.,  1984,  "Integration  of  Methods  for  the  Segmentation  of  Aerial  Photographs  ’. 
Photogrammetria.  vol.  39.  no.  3 

Kirsch,  R.A.,  1971,  "Computer  Determination  of  the  Constituent  Structure  of  Biological  Images”,  Computers 
and  Biomedical  Research,  vol.  4,  no.  3 

Klensberg,  A.  and  Wells,  D.,  1986.  "High  Precision  Differential  GPS  Navigation",  Symp.  on  Position. 
Location  and  Navigation.  Las  Vegas.  NV 

Konecny,  G.,  1970,  "Metric  Problems  in  Remote  Sensing".  Publ.  of  Inti.  Inst,  for  Aerial  Surveying  and 
Earth  Sciences  (1TC),  series  A,  no.  50 

Kropatsch,  W.  and  Leberl,  F.W.,  1981,  Automated  Registration  of  Scanned  Satellite  Imagery  with  a  Digital 


Map  Data  Base.  Department  for  Image  Processing  and  Computer  Mapping,  Graz  Research  Center,  Graz, 
Austria 

Leberl,  F.W.,  1972,  "On  Model  Formation  with  Remote  Sensing  Imagery",  Oestrr.  Zeitschrift  fuer 
Vermessungswesen  und  Photogrammetrie,  no.  2 

Leberl,  F.W.,  1983,  "Photogrammetrie  Aspects  of  Remote  Sensing  with  Imaging  Radar",  Remote  Sensing 
Reviews,  vol.  I,  no.  1 

Leberl,  F.W.  and  Kropatsch,  W.,  1979.  Map-guided  Automatic  Analysis  of  Digital  Satellite  Images, 


Mitteilungen  der  Geodaetischen  Institute  der  Tcchnischen  Universitaet  in  Graz,  folge  33 

Leberl,  F.W.  and  Rnnzinger,  H.,  1982,  Registration  of  Digitized  Aerial  Photography  with  a  Digital  Map 


Data  Base.  Institute  for  Applied  Geodesy  and  Photograinmetry,  Technical  University  of  Graz,  Graz,  Austria 

Logan,  T.L.,  1981,  "A  Database  Approach  for  Prediction  of  Deforestation-Induced  Mass  Wasting  Events", 
Proc.  47th  Ann.  Mtg.  of  the  Amer.  Soc.  Photogrammetry 

Lngnanl,  J.,  1982,  "The  Digitized  Features  -  A  New  Source  of  Control",  Inti.  Archives  of  Photogrammetry. 
vol.  24-III/2 

Marr,  D.  and  Hildreth,  E.,  1980,  "Theory  of  Edge  Detection”,  Proc.  R.  Soc.  London  -  B.  vol.  207 
Marr,  D.  and  Poggio,  T.,  1976.  "Cooperative  Computation  of  Stereo  Disparity".  IEEE-PAMI.  vol,  2,  no.  I 


m —  M  A  *  m  *  mm  *  m  a  "  m  m  "  m  "  > ^  ta»  *  B  m  m  4  ■  f  N  w  ■*  m  •  %  "  «  *  »  ^  ■  *  •  -*  " 


McConnell,  R,,  1987,  "Matching  of  Dissimilar  Radar  Images  Using  Marr-Hildreth  Zero-crossings".  Proc. 
1987  Conf.  Amer.  Soc.  Photogrammetrv.  29  Mnr-3  Apr  1987,  Baltimore.  N1D  (to  lie  published) 


McKeown,  D.M.,  1982,  "Concept  Maps",  Proc.  1982  DARPA  linage  Understanding  Workshop.  Stanford 
Univ..  CA 

McKeown,  D.M.,  1983,  "MAPS:  The  Organization  of  a  Spatial  Database  System  Using  Imagery,  Terrain 
and  Map  Data”,  DARPA  Image  Understanding  Workshop.  June  1983,  Crystal  City,  VA 

McKeown,  D.M,,  1984,  "Spatial  Database  Research  at  CMU".  IEEE  1984  Proc  of  Trends  and  Applications 

McKeown,  D.M.  and  Denlinger,  J.L.,  1984,  "Map-guided  Feature  Extraction  from  Aerial  Imagery",  IEEE 
1984  Proc.  of  Workshop  on  Computer  Vision  Representation  and  Control 

McKeown,  D.M.  and  McDermott,  J.,  1983,  "Toward  Expert  Systems  for  Photo  Interpretation",  IEEE  1983 
Proc.  of  Trends  and  Applications 

McKeown,  et  al,  1984,  "Rule  Based  Interpretation  of  Aerial  Imagery",  IEEE  1984  Proc.  of  Workshop  on 
Principles  of  Knowledge-based  Systems 

Medioni,  G.  and  Nevatia,  R.,  1984.  "Matching  Images  Using  Linear  Features",  IEEE-PAMI,  vol.  6,  no.  6 

Medioni,  G.  and  Nevatia,  R.,  1985,  "Segment-Based  Stereo  Matching".  Comp.  Vision.  Graphics,  and 
Image  Proc..  vol. 31 

Meier,  E.  and  Nuesch,  D.,  1986.  "Geometrische  Entzerrung  von  Bildern  Orbitgestutztcr  SAR-Systeme", 
Bildmessung  und  Luftbildwesen.  54  (1986),  Heft  5 

Milgratn,  D.  and  Rosenfeld,  A.,  1978,  "Algorithms  and  Hardware  Technology  for  Image  Recognition", 
Contract  Report,  U.S.Army  Night  Vision  Laboratory,  Ft.  Belvoir,  VA 

Moklitarian,  F.  and  Mackworth,  A.,  1986,  "Scale-Based  Description  and  Recognition  of  Planar  Curves  and 
Two-Dimensional  Shapes",  IEEE-PAMI,  vol.  8,  no.  I 

Moravec,  H.P.,  1977,  "Towards  Automatic  Visual  Object  Avoidance",  Proc.  Fifth  Inti,  Joint  Conf.  Art,  Int. 

Nack,  M.L.,  1975,  "Temporal  Registration  of  Muliispcclral  Digital  Satellite  Images  Using  Their  Edge 
Images".  AAS/AIAA  Aslrodynamics  Specialist  Conference,  Nassau.  Bahamas,  paper  no.  AAS-75-104 


WWW 


TO 


JI»J!P.!H.II 


v*-'  ^ W  vw  w  v’CAtV.  v  %■*»  ^n  w  v^.w,"in  v  Hr  v  vw 


£ 


s 


v 

i 


* 

ii 

L> 


ft 


iv 

LV 

r-.-1 

ft 


,-\ 

■,: 

\] 

eS 


Nelson,  R.C.  and  Samel.  H..  1986.  "A  Consistent  Hierarchical  Representation  for  Vector  Data",  Computer 
Graphics,  vol.  20.  no.  4  (Proc.  of  S1GGRAPI1  86.  Dallas,  TX.  Aug  1986 

Nevatia,  R.,  1975,  "Object  Boundary  Determination  in  a  Textured  Environment".  Proc.  ACM  '75  Conf, . 
Minneapolis.  MN 

Pnderes,  F.C.,  el  al,  1984.  "Rectification  of  Single  and  Multiple  Frames  of  Satellite  Scanner  Imagery  Using 
Points  and  Edges  as  Control".  NASA  Svmp.  on  Math.  Pattern  Recog,  and  Image  Analysis.  Houston.  TX 

Puccinelli,  E.,  1976.  "Ground  Location  of  Satellite  Scanner  Data",  Pliotgrammetric  Eng,  and  Remote 
Sensing,  vol.  42.  no.  4 

Quam,  L.H.,  1971,  Computer  Comparison  of  Pictures.  Al  Memo  144,  Comp.  Sci.  Dept.,  Stanford  Univ., 
Palo  Alto,  CA 

Raetzsch,  H.,  1985.  Die  Entwicklung  einer  Datcnstniktttr  fuer  die  Speichcrung  und  Manipulation  Digitaler 
Technischer  Graphik.  Dissertation,  Technische  Universitaet  Graz,  Austria 

Ranzinger,  H.,  1985,  Correcting  a  Digital  Map  Data  Base  with  Scene  Analysis  Concepts  and  Methods. 
Institute  for  Image  Processing  and  Computer  Graphics,  Graz  Research  Center,  Graz,  Austria 

Ranzinger,  M.  and  Ranzinger,  II.,  1984.  "Combinations  of  Remote  Sensing  Data  with  a  Digital  Map  Data 
Base*.  Proc:  Integrated  Approaches  in  Remote  Sensing.  8-11  Apr  1984.  Guildford,  UK 

Roberts,  L.G.,  1965,  "Machine  Perception  in  Three-Dimensional  Solids",  in  Optical  and  Electro-Optical 
Information  Processing,  (Tippett,  et  al,  ed.),  MIT  Press 

Robinson,  V.,  et  al,  1986.  Expert  Systems  Applied  to  Problems  in  Geographic  Information  Systems.  Report 
No.  52,  Univ.  of  Maine  at  Orono,  Dept,  of  Civil  Engineering 

van  Roessel,  J.  and  de  Godoy,  R.,  1974,  "SLAR  Mosaics  for  Project  RADAM",  Photogrammetric  Eng., 
vol.  XL 

Samet,  H.,  1984,  The  Quadtree  and  Related  Hierarchical  Data  Structures.  Computing  Surveys,  vol.  16.  no. 

2 

Samet,  H.  and  Webber,  R.E.,  1984,  On  Encoding  Boundaries  with  Quadtrees.  1EEE-PAMI.  vol.  6,  no.  3 


69 


Sobel,  1973.  in  Pattern  Classification  mid  Scene  Analysis.  (Dudn  and  Hart.  eds  ).  Wiley 


Sties,  M.,  et  al,  1977.  Organisation  of  Object  Data  for  an  Image  Processing  System.  FIM-Iustitute. 

Karlsruhe.  FRG 

Tenetibaunt,  M.,  et  ai,  1978,  A  Scene  Analysis  Approach  to  Remote  Sensing.  Tech.  Note  No.  173. 

Artificial  Intelligence  Center.  Stanford  Research  Institute.  CA  ] 

Wiejak,  J.S.,  1985.  "Convolution  with  Separable  Masks  for  Early  Image  Processing",  Comp,  Vision. 

Graphics,  and  Image  Proc.,  vol  32 

i 

t 

i 

\ 

i 


j 

« 

« 

< 

I 

A 

I 

I 


Frank,  A.U.,  1984,  " Requirement  for  Database  Systems  Suitable  to  Manage  Large  Spatial  Databases",  Inti. 
Symp.  on  Spatial  Data  Handling.  Zurich.  Switzerland.  Aug  1984 

Frank,  A.U.  and  Kuhn,  YV.,  1986,  "Cell  Graphs:  A  Provably  Correct  Method  for  the  Storage  of  Geometry^. 
2nd  lntl.  Symp.  on  Spatial  Data  Handling.  Seattle,  WA.  July  1986 

Gilmore,  J.,  1985.  "Artificial  Intelligence  in  Image  Processing",  Soc.  Photo-optical  Instr.  Engin..  vol.  528 

Mark,  D.  and  Abel,  D.,  1985.  "Linear  Quadtrees  from  Vector  Representations  of  Polygons".  IEEE-PAMI. 
vol.  7,  no.  3 

Nagy,  G.  and  YVagle,  S.,  1979.  "Geographic  Data  Processing*,  Computing  Surveys,  vol.  1 1,  no.  2 

Samel,  H.,  1980a.  "Region  Representation:  Quadtrees  from  Boundary  Codes",  Comm,  of  the  ACM,  vol. 
23,  no.  3 

Samet,  H.,  1980b,  "Region  Representation:  Quadtrees  from  Binary  Arrays",  Comp.  Graphics  and  Image 
Proc..  vol.  13 

Samet,  II.  and  Rosenfeld,  A.,  1982,1983,1984.  Application  of  Hierarchical  Data  Structures  to  Geographical 
Information  Systems  (Phases  1. II. III).  Reports  No.  ETL-301,  ETL-337,  ETL-376,  US  Army  Engineer 
Topographic  Laboratories 


71 


-*  V  m*  %■*  _•  'V  ' 

V  A  A  AV. 


